Electronic device and method of operation thereof

ABSTRACT

Various embodiments of the disclosure provide a method and apparatus for processing a voice command in an electronic device. According to various embodiments of the disclosure, the electronic device includes a microphone, a memory, and a processor operatively coupled to the microphone and the memory. The processor may be configured to wake up on the basis of detection of a voice call command, calculate a score related to recognition of the voice call command, share the score with an external device, decide whether to execute a voice command on the basis of the score, and process the voice command on the basis of the decision result. Various embodiments are possible.

This application is the U.S. national phase of International ApplicationNo. PCT/KR2017/011504 filed Oct. 18, 2017 which designated the U.S. andclaims priority to KR Patent Application No. 10-2017-0006047 filed Jan.13, 2017, the entire contents of each of which are hereby incorporatedby reference.

TECHNICAL FIELD

Various embodiments of the disclosure relate to a method and apparatusfor processing a voice command in an electronic device.

BACKGROUND ART

Recently, with the development of digital technologies, various types ofelectronic devices are widely used such as a mobile communicationterminal, a smart phone, a tablet Personal Computer (PC), a notebook, aPersonal Digital Assistant (PDA), a wearable device, a digital camera, apersonal computer, or the like.

Recently, various voice services (e.g., a voice recognition function)are developed and provided in the electronic device on the basis of avoice recognition technique. According to an embodiment, the electronicdevice provides a variety of life convenience services such as mobilesearch, schedule management, dialing, memo, or music playback based on avoice command of a user.

Summary

However, the user may use a plurality of electronic devices, and thesame voice call command may be registered in the plurality of electronicdevices. In this case, since the voice call command is simultaneouslyrecognized in the plurality of electronic devices, not only anelectronic device intended by the user but also a plurality of otherelectronic devices can be driven. As a result, voice commands for user'sutterance are processed by electronic devices even if the electronicdevices are not intended by the user, thereby causing userinconvenience. For example, a case may be assumed where the voice callcommand is equally registered as “Hi, ABC” to the plurality ofelectronic devices, and the user searches for a food recipe. Accordingto an embodiment, the user may utter “Hi, ABC, recommend a good foodrecipe for summer”. In this case, all of the plurality of electronicdevices (e.g., a smart phone, a refrigerator, a TV, etc.) may wake up bythe voice call command “Hi, ABC” to search for the food reciperedundantly, and may provide the user with a redundant result.

Various embodiments disclose a method and apparatus for processing voicecommands based on an electronic device intended by a user of a pluralityof electronic devices.

An electronic device according to various embodiments of the disclosureincludes a microphone, a memory, and a processor operatively coupled tothe microphone and the memory. The processor may be configured to wakeup on the basis of detection of a voice call command, calculate a scorerelated to recognition of the voice call command, share the score withan external device, decide whether to execute a voice command on thebasis of the score, and process the voice command on the basis of thedecision result.

A method of operating an electronic device according to variousembodiments may include waking up on the basis of detection of a voicecall command, calculating a score related to recognition of the voicecall command, sharing the score with an external device, decidingwhether to execute a voice command on the basis of the score, andprocessing the voice command on the basis of the decision result.

Various embodiments of the disclosure may include a computer readablerecording medium recording a program for executing the method in theprocessor.

In an electronic device and an operating method thereof according tovarious embodiments, when a user uses the electronic device to execute(use) a function based on a voice command (e.g., a voice recognitionfunction), the voice command can be processed without inconvenience onthe basis of an intended device even if one voice call command isconfigured for the plurality of electronic devices. According to variousembodiments of the disclosure, a voice is used to simply control theelectronic device so that the user can perform a related operation byrecognizing a voice command in the intended electronic device.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a network environment including an electronic deviceaccording to various embodiments of the disclosure;

FIG. 2 is a block diagram of an electronic device according to variousembodiments of the disclosure;

FIG. 3 is a block diagram of a program module according to variousembodiments of the disclosure;

FIG. 4 briefly illustrates a structure of an electronic device accordingto various embodiments of the disclosure;

FIGS. 5A, 5B and 5C illustrate an example of implementing a componentfor processing a voice command in an electronic device according tovarious embodiments of the disclosure;

FIG. 6 illustrates an example of an electronic device according tovarious embodiments of the disclosure.

FIGS. 7 and 8 illustrate an example of an operation of processing avoice command in a system according to various embodiments of thedisclosure;

FIG. 9 illustrates an example of information considered when a score iscalculated in an electronic device according to various embodiments;

FIG. 10 illustrates an operation of processing a voice command in asystem according to various embodiments of the disclosure;

FIG. 11 is a flowchart illustrating an operation of processing a voicecommand in an electronic device according to various embodiment of thedisclosure;

FIG. 12 is a flowchart illustrating an operation of processing a voicecommand in a server according to various embodiment of the disclosure;

FIG. 13 is a flowchart illustrating an operation of processing a voicecommand in a server according to various embodiments of the disclosure;

FIG. 14 is a flowchart illustrating an example of determining a targetdevice for executing a voice command in a server according to variousembodiments of the disclosure;

FIG. 15 is a flowchart illustrating an operation of processing a voicecommand in an electronic device according to various embodiments of thedisclosure;

FIG. 16 illustrates an operation of processing a voice in a systemaccording to various embodiments of the disclosure;

FIG. 17 is a flowchart illustrating an operation of processing a voicecommand in an electronic device according to various embodiments of thedisclosure;

FIG. 18 is a flowchart illustrating an example of determining a targetdevice for executing a voice command in an electronic device accordingto various embodiments of the disclosure; and

FIG. 19 is a flowchart illustrating an example of determining a targetdevice for executing a voice command in an electronic device accordingto various embodiments of the disclosure.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

Hereinafter, various embodiments of the present disclosure are disclosedwith reference to the accompanying drawings. It should be appreciatedthat various embodiments of the present disclosure and the terms usedtherein are not intended to limit the technological features set forthherein to particular embodiments and include various changes,equivalents, and/or replacements for a corresponding embodiment. Withregard to the description of the drawings, similar reference numeralsmay be used to refer to similar or related elements. It is to beunderstood that a singular form of a noun corresponding to an item mayinclude one or more of the things, unless the relevant context clearlyindicates otherwise. As used herein, each of such phrases as “A or B” or“at least one of A and/or B,” may include any one of, or all possiblecombinations of the items enumerated together in a corresponding one ofthe phrases. As used herein, such terms as “1st” and “2nd,” or “first”and “second” may be used to simply distinguish a corresponding componentfrom another, and does not limit the components in other aspect (e.g.,importance or order). It is to be understood that if an element (e.g., afirst element) is referred to, with or without the term “operatively” or“communicatively”, as “coupled with,” “coupled to,” “connected with,” or“connected to” another element (e.g., a second element), it means thatthe element may be coupled with the other element directly (e.g.,wiredly), wirelessly, or via a third element.

The expression “configured to” used in various embodiments of thepresent disclosure may be interchangeably used with “suitable for,”“having the capacity to,” “designed to,” “adapted to,” “made to,” or“capable of” according to the situation, for example. The term“configured to” may not necessarily indicate “specifically designed to”in terms of hardware. Instead, the expression “a device configured to”in some situations may indicate that the device and another device orpart are “capable of.” For example, the expression “a processorconfigured to perform A, B, and C” may indicate a dedicated processor(for example, an embedded processor) for performing a correspondingoperation or a general purpose processor (for example, a centralprocessing unit (CPU) or application processor (AP)) for performingcorresponding operations by executing at least one software programstored in a memory device.

An electronic device according to various embodiments of the presentdisclosure may include at least one of a smartphone, a tablet personalcomputer (PC), a mobile phone, a video telephone, an electronic bookreader, a desktop PC, a laptop PC, a netbook computer, a workstation, aserver, a personal digital assistant (PDA), a portable multimedia player(PMP), a Motion Picture Experts Group (MPEG-1 or MPEG-2) Audio Layer 3(MP3) player, a mobile medical device, a camera, or a wearable device.The wearable device may include at least one of an accessory-type device(e.g., a watch, a ring, a bracelet, an anklet, a necklace, glasses, acontact lens, a head-mounted device (HMD)), a textile- orclothing-integrated-type device (e.g., an electronic apparel), abody-attached-type device (e.g., a skin pad or a tattoo), or abio-implantable-type device (e.g., an implantable circuit). In somevarious embodiments of the present disclosure, an electronic device mayinclude at least one of, for example, a television (TV), a digitalvideo/versatile disc (DVD) player, an audio, a refrigerator, an airconditioner, a cleaner, an oven, a microwave oven, a washing machine, anair cleaner, a set-top box, a home automation control panel, a securitycontrol panel, a media box (e.g., Samsung HomeSync™, Apple TV™, orGoogle TV™), a game console (e.g., Xbox™ or PlayStation™), an electronicdictionary, an electronic key, a camcorder, or an electronic pictureframe.

In other various embodiments of the present disclosure, an electronicdevice may include at least one of various medical devices (e.g.,various portable medical measurement devices (e.g., a blood glucosemeasuring device, a heart rate measuring device, a blood pressuremeasuring device, a body temperature measuring device, or the like), amagnetic resonance angiography (MRA), a magnetic resonance imaging(MRI), a computed tomography (CT), a scanner, an ultrasonic device, orthe like), a navigation device, a global navigation satellite system(GNSS), an event data recorder (EDR), a flight data recorder (FDR), avehicle infotainment device, electronic equipment for vessels (e.g., anavigation system, a gyrocompass, or the like), avionics, a securitydevice, a head unit for a vehicle, an industrial or home robot, anautomatic teller machine (ATM), a point of sales (POS) device of astore, or an Internet of things (IoT) device (e.g., a light bulb,various sensors, an electric or gas meter, a sprinkler, a fire alarm, athermostat, a streetlamp, a toaster, exercise equipment, a hot watertank, a heater, a boiler, or the like). According to various embodimentsof the present disclosure, an electronic device may include at least oneof a part of furniture or a building/structure, an electronic board, anelectronic signature receiving device, a projector, or a measuringinstrument (e.g., a water meter, an electricity meter, a gas meter, awave meter, or the like). An electronic device may be one or morecombinations of the above-mentioned devices. An electronic deviceaccording to some various embodiments of the present disclosure may be aflexible device. An electronic device according to an embodiment of thepresent disclosure is not limited to the above-mentioned devices, andmay include new electronic devices with the development of newtechnology. The term “user” used herein may refer to a person who usesan electronic device or may refer to a device (e.g., an artificialintelligence electronic device) that uses an electronic device.

FIG. 1 illustrates an electronic device in a network environmentaccording to an embodiment of the present disclosure.

An electronic device 101 in a network environment 100 according tovarious embodiments of the present disclosure will be described withreference to FIG. 1 . The electronic device 101 may include a bus 110, aprocessor 120, a memory 130, an input/output interface 150, a display160, and a communication interface 170. In various embodiments of thepresent disclosure, at least one of the foregoing elements may beomitted or another element may be added to the electronic device 101.

The bus 110 may include a circuit for connecting the above-mentionedelements 110 to 170 to each other and transferring communications (e.g.,control messages and/or data) among the above-mentioned elements.

The processor 120 may include at least one of a central processing unit(CPU), an application processor (AP), or a communication processor (CP).The processor 120 may perform data processing or an operation related tocommunication and/or control of at least one of the other elements ofthe electronic device 101.

The memory 130 may include a volatile memory and/or a nonvolatilememory. The memory 130 may store instructions or data related to atleast one of the other elements of the electronic device 101. Accordingto an embodiment of the present disclosure, the memory 130 may storesoftware and/or a program 140. The program 140 may include, for example,a kernel 141, a middleware 143, an application programming interface(API) 145, and/or an application program (or an application) 147. Atleast a portion of the kernel 141, the middleware 143, or the API 145may be referred to as an operating system (OS).

The kernel 141 may control or manage system resources (e.g., the bus110, the processor 120, the memory 130, or the like) used to performoperations or functions of other programs (e.g., the middleware 143, theAPI 145, or the application program 147). Furthermore, the kernel 141may provide an interface for allowing the middleware 143, the API 145,or the application program 147 to access individual elements of theelectronic device 101 in order to control or manage the systemresources.

The middleware 143 may serve as an intermediary so that the API 145 orthe application program 147 communicates and exchanges data with thekernel 141. Furthermore, the middleware 143 may handle one or more taskrequests received from the application program 147 according to apriority order. For example, the middleware 143 may assign at least oneapplication program 147 a priority for using the system resources (e.g.,the bus 110, the processor 120, the memory 130, or the like) of theelectronic device 101. For example, the middleware 143 may handle theone or more task requests according to the priority assigned to the atleast one application. The API 145, which is an interface for allowingthe application 147 to control a function provided by the kernel 141 orthe middleware 143, may include, for example, at least one interface orfunction (e.g., instructions) for file control, window control, imageprocessing, character control, or the like.

The input/output interface 150 may serve to transfer an instruction ordata input from a user or another external device to (an)otherelement(s) of the electronic device 101. Furthermore, the input/outputinterface 150 may output instructions or data received from (an)otherelement(s) of the electronic device 101 to the user or another externaldevice.

The display 160 may include, for example, a liquid crystal display(LCD), a light-emitting diode (LED) display, an organic light-emittingdiode (OLED) display, an active matrix OLED (AMOLED), amicroelectromechanical systems (MEMS) display, or an electronic paperdisplay. The display 160 may present various content (e.g., a text, animage, a video, an icon, a symbol, or the like) to the user. The display160 may include a touch screen, and may receive a touch, gesture,proximity or hovering input from an electronic pen or a part of a bodyof the user.

The communication interface 170 may set communications between theelectronic device 101 and an external device (e.g., a first externalelectronic device 102, a second external electronic device 104, or aserver 106). For example, the communication interface 170 may beconnected to a network 162 via wireless communications or wiredcommunications so as to communicate with the external device (e.g., thesecond external electronic device 104 or the server 106).

The wireless communications may employ at least one of cellularcommunication protocols such as long-term evolution (LTE), LTE-advance(LTE-A), code division multiple access (CDMA), wideband CDMA (WCDMA),universal mobile telecommunications system (UMTS), wireless broadband(WiBro), or global system for mobile communications (GSM). The wirelesscommunications may include, for example, at least one of wirelessfidelity (Wi-Fi), wireless gigabit alliance (WiGig), Bluetooth,Bluetooth low energy (BLE), Zigbee, near field communication (NFC),magnetic secure transmission, radio frequency (RF), or body area network(BAN). The wireless communications according to an embodiment includeGNSS. The GNSS may include, for example, global positioning system(GPS), global navigation satellite system (GLONASS), BeiDou navigationsatellite system (BeiDou), or Galileo, the European globalsatellite-based navigation system. Hereinafter, the term “GPS” and theterm “GNSS” may be interchangeably used.

The wired communications may include at least one of universal serialbus (USB), high definition multimedia interface (HDMI), recommendedstandard 232 (RS-232), power line communication, plain old telephoneservice (POTS), or the like.

The network 162 may include at least one of telecommunications networks,for example, a computer network (e.g., local area network (LAN) or widearea network (WAN)), the Internet, or a telephone network.

The types of the first external electronic device 102 and the secondexternal electronic device 104 may be the same as or different from thetype of the electronic device 101. A portion or all of operationsperformed in the electronic device 101 may be performed in one or moreother electronic devices (e.g., the first electronic device 102, thesecond external electronic device 104, or the server 106). When theelectronic device 101 should perform a certain function or serviceautomatically or in response to a request, the electronic device 101 mayrequest at least a portion of functions related to the function orservice from another device (e.g., the first electronic device 102, thesecond external electronic device 104, or the server 106) instead of orin addition to performing the function or service for itself. The otherelectronic device (e.g., the first electronic device 102, the secondexternal electronic device 104, or the server 106) may perform therequested function or additional function, and may transfer a result ofthe performance to the electronic device 101. The electronic device 101may use a received result itself or additionally process the receivedresult to provide the requested function or service. To this end, forexample, a cloud computing technology, a distributed computingtechnology, or a client-server computing technology may be used.

The server 106 may include, for example, at least one of voicerecognition server, voice service providing server, data server,searching server, payment server, card company server, bank server,authentication server, application server, management server,integration server, provider server (or telecommunications operatorserver), content server, internet server, or cloud server.

FIG. 2 is a block diagram illustrating an electronic device according toan embodiment of the present disclosure.

An electronic device 201 may include, for example, a part or theentirety of the electronic device 101 illustrated in FIG. 1 . Theelectronic device 201 may include at least one processor (e.g., AP) 210,a communication module 220, a subscriber identification module (SIM)224, a memory 230, a sensor module 240, an input device 250, a display260, an interface 270, an audio module 280, a camera module 291, a powermanagement module 295, a battery 296, an indicator 297, and a motor 298.In various embodiments of the present disclosure, the electronic device201 may include more element or less element than the foregoing elementsbecause the foregoing elements described in FIG. 2 are not essential. Invarious embodiments of the present disclosure, the electronic device 201may not include some of the foregoing elements. In various embodimentsof the present disclosure, the elements of the electronic device 201 areembedded in a housing (or bezel, body) of the electronic device 201, orformed on exterior thereof.

The processor 210 may run an operating system or an application programso as to control a plurality of hardware or software elements connectedto the processor 210, and may process various data and performoperations. The processor 210 may be implemented with, for example, asystem on chip (SoC). According to an embodiment of the presentdisclosure, the processor 210 may further include a graphic processingunit (GPU) and/or an image signal processor. The processor 210 mayinclude at least a portion (e.g., a cellular module 221) of the elementsillustrated in FIG. 2 . The processor 210 may load, on a volatilememory, an instruction or data received from at least one of otherelements (e.g., a nonvolatile memory) to process the instruction ordata, and may store various data in a nonvolatile memory.

The communication module 220 may have a configuration that is the sameas or similar to that of the communication interface 170 of FIG. 1 . Thecommunication module 220 may include, for example, a cellular module221, a Wi-Fi module 223, a Bluetooth (BT) module 225, a GNSS module 227,a NFC module 228, and a radio frequency (RF) module 229. Thecommunication module 220 may further include, for example, WiGig module.According to an embodiment, the WiFi module 223 and the WiGig module maybe implemented in a single integrated chip.

The cellular module 221 may provide, for example, a voice call service,a video call service, a text message service, or an Internet servicethrough a communication network. The cellular module 221 may identifyand authenticate the electronic device 201 in the communication networkusing the subscriber identification module 224 (e.g., a SIM card). Thecellular module 221 may perform at least a part of functions that may beprovided by the processor 210. The cellular module 221 may include acommunication processor (CP). According to an embodiment of the presentdisclosure, at least a part (e.g., two or more) of the cellular module221, the Wi-Fi module 223, the Bluetooth module 225, the GNSS module227, and the NFC module 228 may be included in a single integrated chip(IC) or IC package.

The RF module 229 may transmit/receive, for example, communicationsignals (e.g., RF signals). The RF module 229 may include, for example,a transceiver, a power amp module (PAM), a frequency filter, a low noiseamplifier (LNA), an antenna, or the like. According to anotherembodiment of the present disclosure, at least one of the cellularmodule 221, the Wi-Fi module 223, the Bluetooth module 225, the GNSSmodule 227, or the NFC module 228 may transmit/receive RF signalsthrough a separate RF module.

The SIM 224 may include, for example, an embedded SIM and/or a cardcontaining the subscriber identity module, and may include uniqueidentification information (e.g., an integrated circuit card identifier(ICCID)) or subscriber information (e.g., international mobilesubscriber identity (IMSI)).

The memory 230 (e.g., the memory 130) may include, for example, aninternal memory 232 or an external memory 234. The internal memory 232may include at least one of a volatile memory (e.g., a dynamic RAM(DRAM), a static RAM (SRAM), a synchronous dynamic RAM (SDRAM), or thelike), a nonvolatile memory (e.g., a one-time programmable ROM (OTPROM),a programmable ROM (PROM), an erasable and programmable ROM (EPROM), anelectrically erasable and programmable ROM (EEPROM), a mask ROM, a flashROM, a flash memory (e.g., a NAND flash memory, a NOR flash memory, orthe like)), a hard drive, or a solid state drive (SSD). The externalmemory 234 may include a flash drive such as a compact flash (CF), asecure digital (SD), a Micro-SD, a Mini-SD, an extreme digital (xD), aMultiMediaCard (MMC), a memory stick, or the like. The external memory234 may be operatively and/or physically connected to the electronicdevice 201 through various interfaces.

The sensor module 240 may, for example, measure physical quantity ordetect an operation state of the electronic device 201 so as to convertmeasured or detected information into an electrical signal. The sensormodule 240 may include, for example, at least one of a gesture sensor240A, a gyro sensor 240B, a barometric pressure sensor 240C, a magneticsensor 240D, an acceleration sensor 240E, a grip sensor 240F, aproximity sensor 240G, a color sensor 240H (e.g., a red/green/blue (RGB)sensor), a biometric sensor 240I, a temperature/humidity sensor 240J, anillumination sensor 240K, or an ultraviolet (UV) sensor 240M.Additionally or alternatively, the sensor module 240 may include, forexample, an olfactory sensor (E-nose sensor), an electromyography (EMG)sensor, an electroencephalogram (EEG) sensor, an electrocardiogram (ECG)sensor, an infrared (IR) sensor, an iris recognition sensor, and/or afingerprint sensor. The sensor module 240 may further include a controlcircuit for controlling at least one sensor included therein. In somevarious embodiments of the present disclosure, the electronic device 201may further include a processor configured to control the sensor module240 as a part of the processor 210 or separately, so that the sensormodule 240 is controlled while the processor 210 is in a sleep state.

The input device 250 may include, for example, a touch panel 252, a(digital) pen sensor 254, a key 256, or an ultrasonic input device 258.The touch panel 252 may employ at least one of capacitive, resistive,infrared, and ultraviolet sensing methods. The touch panel 252 mayfurther include a control circuit. The touch panel 252 may furtherinclude a tactile layer so as to provide a haptic feedback to a user.

The (digital) pen sensor 254 may include, for example, a sheet forrecognition which is a part of a touch panel or is separate. The key 256may include, for example, a physical button, an optical button, or akeypad. The ultrasonic input device 258 may sense ultrasonic wavesgenerated by an input tool through a microphone 288 so as to identifydata corresponding to the ultrasonic waves sensed. According to variousembodiments, the input device 250 may include an electronic pen.According to various embodiments, the input device may be implemented toreceive a force touch.

The display 260 (e.g., the display 160) may include a panel 262, ahologram device 264, a projector 266, and/or a control circuit forcontrolling them.

The panel 262 may be, for example, flexible, transparent, or wearable.The panel 262 and the touch panel 252 may be integrated into a singlemodule. The panel 262 may include a force sensor for measuring strengthof pressure according to a touch of a user. The force sensor may beimplemented with the touch panel 252 or may be implemented independentlyin a form of one or more sensors.

The hologram device 264 may display a stereoscopic image in a spaceusing a light interference phenomenon. The projector 266 may projectlight onto a screen so as to display an image. The screen may bedisposed in the inside or the outside of the electronic device 201.

The interface 270 may include, for example, an HDMI 272, a USB 274, anoptical interface 276, or a D-subminiature (D-sub) 278. The interface270, for example, may be included in the communication interface 170illustrated in FIG. 1 . Additionally or alternatively, the interface 270may include, for example, a mobile high-definition link (MHL) interface,an SD card/multimedia card (MMC) interface, or an infrared dataassociation (IrDA) interface.

The audio module 280 may convert, for example, a sound into anelectrical signal or vice versa. At least a portion of elements of theaudio module 280 may be included in the input/output interface 150illustrated in FIG. 1 . The audio module 280 may process soundinformation input or output through a speaker 282, a receiver 284, anearphone 286, or the microphone 288.

The camera module 291 is, for example, a device for shooting a stillimage or a video. According to an embodiment of the present disclosure,the camera module 291 may include at least one image sensor (e.g., afront sensor or a rear sensor), a lens, an image signal processor (ISP),or a flash (e.g., an LED or a xenon lamp).

The power management module 295 may manage power of the electronicdevice 201. According to an embodiment of the present disclosure, thepower management module 295 may include a power management integratedcircuit (PMIC), a charger integrated circuit (IC), or a battery orgauge. The PMIC may employ a wired and/or wireless charging method. Thewireless charging method may include, for example, a magnetic resonancemethod, a magnetic induction method, an electromagnetic method, or thelike. An additional circuit for wireless charging, such as a coil loop,a resonant circuit, a rectifier, or the like, may be further included.The battery gauge may measure, for example, a remaining capacity of thebattery 296 and a voltage, current or temperature thereof while thebattery is charged. The battery 296 may include, for example, arechargeable battery and/or a solar battery.

The indicator 297 may display a specific state of the electronic device201 or a part thereof (e.g., the processor 210), such as a bootingstate, a message state, a charging state, or the like. The motor 298 mayconvert an electrical signal into a mechanical vibration, and maygenerate a vibration or haptic effect. Although not illustrated, aprocessing device (e.g., a GPU) for supporting a mobile TV may beincluded in the electronic device 201. The processing device forsupporting a mobile TV may process media data according to the standardsof digital multimedia broadcasting (DMB), digital video broadcasting(DVB), MediaFLO™, or the like.

Each of the elements described herein may be configured with one or morecomponents, and the names of the elements may be changed according tothe type of an electronic device. In various embodiments of the presentdisclosure, an electronic device may include at least one of theelements described herein, and some elements may be omitted or otheradditional elements may be added. Furthermore, some of the elements ofthe electronic device may be combined with each other so as to form oneentity, so that the functions of the elements may be performed in thesame manner as before the combination.

FIG. 3 is a block diagram illustrating a program module according to anembodiment of the present disclosure.

According to an embodiment, a program module 310 (e.g., the program 140)may include an operating system (OS) for controlling a resource relatedto an electronic device (e.g., the electronic device 101, 201) and/orvarious applications (e.g., the application program 147) running on theOS. The operating system may be, for example, Android™, iOS™, Windows™,Symbian™, Tizen™, or Bada™.

Referring to FIG. 3 , the program module 310 may include a kernel 320(e.g., the kernel 141), a middleware 330 (e.g., the middleware 143), anAPI 360 (e.g., the API 145), and/or an application 370 (e.g., theapplication program 147). At least a part of the program module 310 maybe preloaded on an electronic device or may be downloaded from anexternal electronic device (e.g., the first electronic device 102, thesecond external electronic device 104, or the server 106).

The kernel 320 may include, for example, a system resource manager 321or a device driver 323. The system resource manager 321 may performcontrol, allocation, or retrieval of a system resource. According to anembodiment of the present disclosure, the system resource manager 321may include a process management unit, a memory management unit, a filesystem management unit, or the like. The device driver 323 may include,for example, a display driver, a camera driver, a Bluetooth driver, ashared memory driver, a USB driver, a keypad driver, a Wi-Fi driver, anaudio driver, or an inter-process communication (IPC) driver. Themiddleware 330, for example, may provide a function that theapplications 370 require in common, or may provide various functions tothe applications 370 through the API 360 so that the applications 370may efficiently use limited system resources in the electronic device.

According to an embodiment of the present disclosure, the middleware 330may include at least one of a runtime library 335, an applicationmanager 341, a window manager 342, a multimedia manager 343, a resourcemanager 344, a power manager 345, a database manager 346, a packagemanager 347, a connectivity manager 348, a notification manager 349, alocation manager 350, a graphic manager 351, and a security manager 352.

The runtime library 335 may include, for example, a library module thata complier uses to add a new function through a programming languagewhile the application 370 is running. The runtime library 335 mayperform a function for input/output management, memory management, or anarithmetic function.

The application manager 341 may mange, for example, a life cycle of atleast one of the applications 370. The window manager 342 may manage aGUI resource used in a screen. The multimedia manager 343 may recognizea format required for playing various media files and may encode ordecode a media file using a codec matched to the format. The resourcemanager 344 may manage a resource such as a source code, a memory, or astorage space of at least one of the applications 370.

The power manager 345, for example, may operate together with a basicinput/output system (BIOS) to manage a battery or power and may providepower information required for operating the electronic device.

The database manager 346 may generate, search, or modify a database tobe used in at least one of the applications 370. The package manager 347may manage installation or update of an application distributed in apackage file format.

The connectivity manger 348 may manage wireless connection of Wi-Fi,Bluetooth, or the like. The notification manager 349 may display ornotify an event such as message arrival, appointments, and proximityalerts in such a manner as not to disturb a user. The location manager350 may manage location information of the electronic device. Thegraphic manager 351 may manage a graphic effect to be provided to a useror a user interface related thereto. The security manager 352 mayprovide various security functions required for system security or userauthentication.

According to an embodiment of the present disclosure, in the case inwhich an electronic device (e.g., the electronic device 101) includes aphone function, the middleware 330 may further include a telephonymanager for managing a voice or video call function of the electronicdevice. The middleware 330 may include a middleware module for forming acombination of various functions of the above-mentioned elements. Themiddleware 330 may provide a module specialized for each type of anoperating system to provide differentiated functions. Furthermore, themiddleware 330 may delete a part of existing elements or may add newelements dynamically.

The API 360 (e.g., the API 145) which is, for example, a set of APIprogramming functions may be provided in different configurationsaccording to an operating system. For example, in the case of Android oriOS, one API set may be provided for each platform, and, in the case ofTizen, at least two API sets may be provided for each platform.

The application 370, for example, may include at least one applicationcapable of performing functions such as a home 371, a dialer 372, anSMS/MMS 373, an instant message (IM) 374, a browser 375, a camera 376,an alarm 377, a contact 378, a voice dial 379, an e-mail 380, a calendar381, a media player 382, an album 383, a watch 384, health care (e.g.,measure an exercise amount or blood sugar), or environmental informationprovision (e.g., provide air pressure, humidity, or temperatureinformation).

According to an embodiment of the present disclosure, the application370 may include an information exchange application for supportinginformation exchange between the electronic device (e.g., the electronicdevice 101) and an external electronic device (e.g., the firstelectronic device 102 or the second external electronic device 104). Theinformation exchange application may include, for example, anotification relay application for relaying specific information to theexternal electronic device or a device management application formanaging the external electronic device.

For example, the notification relay application may have a function forrelaying, to an external electronic device, notification informationgenerated in another application of the electronic device. Furthermore,the notification relay application may receive notification informationfrom the external electronic device and may provide the receivednotification information to the user.

The device management application, for example, may manage (e.g.,install, delete, or update) at least one function (e.g., turn-on/turnoff of the external electronic device itself (or some elements) or thebrightness (or resolution) adjustment of a display) of the externalelectronic device communicating with the electronic device, anapplication running in the external electronic device, or a serviceprovided from the external electronic device.

According to an embodiment of the present disclosure, the application370 may include a specified application (e.g., a healthcare applicationof a mobile medical device) according to an attribute of the externalelectronic device.

The application 370 may include an application received from an externalelectronic device. According to various embodiments of the presentdisclosure, at least a part of the program module 310 may be implementedwith software, firmware, hardware, or a combination thereof. At least apart of the program module 310 may include, for example, a module, aprogram, a routine, sets of instructions, or a process for performing atleast one function.

The term “module” used herein may represent, for example, a unitincluding one of hardware, software and firmware or a combinationthereof. The term “module” may be interchangeably used with the terms“unit”, “logic”, “logical block”, “component” and “circuit”. The“module” may be a minimum unit of an integrated component or may be apart thereof. The “module” may be a minimum unit for performing one ormore functions or a part thereof. The “module” may be implementedmechanically or electronically. For example, the “module” may include atleast one of an application-specific integrated circuit (ASIC) chip, afield-programmable gate array (FPGA), and a programmable-logic devicefor performing some operations, which are known or will be developed.

At least a part of devices (e.g., modules or functions thereof) ormethods (e.g., operations) according to various embodiments of thepresent disclosure may be implemented as instructions stored in acomputer-readable storage medium (e.g., the memory 130) in the form of aprogram module. In the case where the instructions are performed by aprocessor (e.g., the processor 120, 210), the processor may performfunctions corresponding to the instructions.

A computer-readable recording medium may include a hard disk, a floppydisk, a magnetic medium (e.g., a magnetic tape), an optical medium(e.g., CD-ROM, digital versatile disc (DVD)), a magneto-optical medium(e.g., a floptical disk), or a hardware device (e.g., a ROM, a RAM, aflash memory, or the like).

The program instructions may include machine language codes generated bycompilers and high-level language codes that can be executed bycomputers using interpreters.

According to various embodiments, the recording medium may include acomputer-readable recording medium to record a program for executingvarious method described below at the processor 120, 210.

A module or a program module according to various embodiments of thepresent disclosure may include at least one of the above-mentionedelements, or some elements may be omitted or other additional elementsmay be added. Operations performed by the module, the program module orother elements according to various embodiments of the presentdisclosure may be performed in a sequential, parallel, iterative orheuristic way. Furthermore, some operations may be performed in anotherorder or may be omitted, or other operations may be added.

Various embodiments of the disclosure provide a method and apparatus forprocessing a voice command (or a voice recognition function, a voiceservice) in an electronic device. According to various embodiments,provided is a method and apparatus for processing a voice command when avoice call command (e.g., a wake-up word or a wake-up command) by whichan electronic device can recognize a start of a voice recognitionfunction is equally registered in a plurality of electronic devices.According to various embodiments, in a multi-device environment in whichthe same voice call command can be recognized, a plurality of electronicdevices may wake up on the basis of the same voice call command andthereafter may automatically determine a master device for executingvoice command recognition and/or feedback. According to variousembodiments, an action (or an operation, a function) depending on thevoice command may be executed based on the master device, or anelectronic device for executing the action may be selected andcontrolled.

An electronic device according to various embodiments may include alldevices using one or more of various processors such as an ApplicationProcessor (AP), a Communication Processor (CP), a Graphic ProcessingUnit (GPU), a Central Processing Unit (CPU), and the like. For example,the electronic device according to various embodiments may include aninformation communication devices, a multimedia device, a wearabledevice, an Internet of Things (IoT) device, or various other devicescorresponding to these devices.

Hereinafter, an operating method and apparatus will be describedaccording to various embodiments of the disclosure with reference to theaccompanying drawings. However, since the various embodiments of thedisclosure are not restricted or limited by the content described below,it should be noted that the disclosure is applicable to the variousembodiments on the basis of embodiments described below. Ahardware-based access method is described for example in the variousembodiments of the disclosure described hereinafter. However, since thevarious embodiments of the disclosure include a technique in whichhardware and software are both used, a software-based access method isnot excluded in various embodiments of the disclosure.

FIG. 4 briefly illustrates a structure of an electronic device accordingto various embodiments of the disclosure.

Referring to FIG. 4 , an electronic device 400 according to variousembodiments may include, for example, a wireless communication unit 410,a user input unit 420, a touch screen 430, an audio processor 440, amemory 450, an interface 460, a camera module 470, a processor 480(e.g., the processor 120 of FIG. 1 , the processor 210 of FIG. 2 ), anda power supplier 490. In various embodiments, the electronic device 400is not necessarily constructed of components of FIG. 4 , and thus thenumber of components thereof may be greater than or less than the numberof components of FIG. 4 .

The wireless communication unit 410 may be constructed identically orsimilarly, for example, to the communication module 220 of FIG. 2 . Thewireless communication unit 410 may include one or more modules forenabling wireless communication between the electronic device 400 and adifferent external electronic device (e.g., the electronic device 102 or104, the server 106). For example, the wireless communication unit 410may include a mobile communication module 411, a Wireless Local AreaNetwork (WLAN) module 413, a short-range communication module 415, alocation calculation module 417, or the like. In various embodiments,the wireless communication unit 410 may include a module for performingcommunication with a neighboring external electronic device (e.g., ashort-range communication module, a long-range communication module, orthe like).

The mobile communication module 411 may be constructed identically orsimilarly, for example, to the cellular module 221 of FIG. 2 . Themobile communication module 411 may transmit/receive a radio signal withrespect to at least one of a base station, an external electronic device(e.g., the different electronic device 104), and various servers on amobile communication network. The radio signal may include a voicesignal, a data signal, or various types of control signals. The mobilecommunication module 411 may transmit a variety of data required for anoperation of the electronic device 400 to an external device (e.g., theserver 106 or the different electronic device 104 or the like) inresponse to a user request.

The WLAN module 413 may be constructed identically or similarly, forexample, to the WiFi module 223 of FIG. 2 . The WLAN module 413 mayindicate a module for establishing a WLAN link with respect to awireless Internet access and an external device (e.g., the differentelectronic device 102 or the server 106 or the like). The WLAN module413 may be placed inside or outside the electronic device 400. Awireless Internet technique may use Wireless Fidelity (WiFi), Wirelessbroadband (Wibro), World interoperability for Microwave access (WiMax),High Speed Downlink Packet Access (HSDPA), millimeter Wave (mmWave), orthe like. The WLAN module 413 may be directly coupled to the electronicdevice 400 or interwork with the external device (e.g., the differentelectronic, etc.) coupled through a network (e.g., the wireless Internetnetwork) (e.g., the network 162 of FIG. 1 ), to transmit a variety ofdata of the electronic device 400 to the outside or receive the datafrom the outside. The WLAN module 413 may maintain an always-on state,or may be turned on/turned off according to a configuration of theelectronic device 400 or a user input.

The short-range communication module 415 may indicate one or moremodules (e.g., the Bluetooth module 225, the NFC module 228, etc.) forperforming short-range communication. A short-range communicationtechnique may use, for example, Bluetooth, Bluetooth Low Energy (BLE),Radio Frequency IDentification (RFID), Infrared Data Association (IrDA),Ultra Wideband (UWB), ZigBee, Near Field Communication (NFC), or thelike. The short-range communication module 415 may transmit a variety ofdata of the electronic device 400 to the outside or receive the datafrom the outside by interworking with the external device (e.g., thedifferent electronic device 102, etc.) coupled to the electronic device400 through a network (e.g., a short-range communication network). Theshort-range communication module 415 may maintain an always-on state, ormay be turned on/turned off according to a configuration of theelectronic device 400 or a user input.

The location calculation module 417 may be constructed identically orsimilarly, for example, to the GNSS module 227 of FIG. 2 . The locationcalculation module 417 is a module for acquiring a location of theelectronic device 400, and a representative example thereof may includea Global Position System (GPS) module. The location calculation module417 may measure the location of the electronic device 400 according to atriangulation principle.

The user input unit 420 may generate input data for controlling theoperation of the electronic device 400 in response to a user input. Theuser input unit 420 may include at least one input means for detectingvarious user inputs. For example, the user input unit 420 may include akey pad, a dome switch, a physical button, a touch pad (staticpressure/electrostatic), jog & shuttle, a sensor (e.g., the sensormodule 240 of FIG. 2 ), or the like.

The user input unit 420 may be partially implemented outside theelectronic device 400 in a button form, and may be partially or entirelyimplemented with a touch panel. The user input unit 420 may receive auser input for initiating various operations (e.g., a voice recognitionfunction, an application execution function, a capture function, a datacreation function, a data reproduction function, etc.) of the electronicdevice 400, and may produce an input signal based on the user input.

The touch screen 430 may indicate an input/output device capable ofsimultaneously performing an input function and a display function.According to various embodiments, the touch screen 430 may include adisplay 431 (e.g., the display 160 of FIG. 1 or the display 260 of FIG.2 ) and a touch detector 433. The touch screen 430 may provide aninput/output interface between the electronic device 400 and the user.For example, the touch screen 430 may deliver a touch input of the userto the electronic device 400, or may play a mediation role for showingan output from the electronic device 400 to the user. The touch screen430 may show a visual output to the user. The visual output may be shownin a form of a text, a graphic, a video, and a combination of them.

The display 431 may display (output) a variety of information processedin the electronic device 400. For example, the display 431 may displayvarious User Interfaces (Uls) or Graphic UI (GUIs) related to using ofthe electronic device 400. According to various embodiments, the display431 may provide various screen configurations to provide the user with afeedback indicating whether it is in an active (or timeout) state duringa voice recognition duration (or period) for processing a voice command.Various displays (e.g., the display 160 of FIG. 1 ) may be used as thedisplay 431. In various embodiments, a curved display (or a bendeddisplay) may be used as the display 431.

The touch detector 433 may be mounted on the display 431, and may detecta user input which is in contact with or approaches to a surface of thetouch screen 430. The user input may include a touch event or aproximity event which is input based on at least one of a single-touch,a multi-touch, a hovering, and an air gesture. In various embodiments,the touch detector 433 may receive a user input for initiating anoperation related to a usage of the electronic device 400, and maygenerate an input signal depending on the user input. The touch detector433 may be configured to convert a change in a pressure applied to aspecific portion of the display 431 or a capacitance or the like whichis generated at a specific portion of the display 431 into an electricalinput signal. The touch detector 433 may detect a location and area inwhich an input tool (e.g., a user's finger, an electronic pen, etc.) isin touch with or approaches to the surface of the display 431. Further,the touch detector 433 may be implemented to be able to detect up to apressure (e.g., a force touch) when a touch is made according to anapplied touch mechanism.

The audio processor 440 may be constructed identically or similarly, forexample, to the audio module 280 of FIG. 2 . The audio processor 440 mayperform a function of transmitting an audio signal input from theprocessor 480 to a speaker (SPK) 441 and delivering an audio signal suchas a voice or the like input from a microphone (MIC) 443 to theprocessor 480. The audio processor 440 may output voice/sound data bytranslating it into an audible sound through the SPK 441 under thecontrol of the processor 480, and may deliver an audio signal such as avoice or the like received from the MIC 443 by translating it into adigital signal.

The SPK 441 may output audio data received from the wirelesscommunication unit 410 or stored in the memory 450. The SPK 441 mayoutput a sound signal related to various operations (functions)performed in the electronic device 400.

The MIC 443 may receive an external sound signal, and may process it aselectrical voice data. Various noise reduction algorithms may beimplemented in the MIC 443 to remove a noise generated in a process ofreceiving the external sound signal. The MIC 443 may manage an input ofaudio streaming such as a voice command (e.g., a voice command forcontrolling activation/deactivation of a voice recognition function orthe like).

The memory 450 (e.g., the memory 130 of FIG. 1 or the memory 230 of FIG.2 ) may store one or more programs executed by the processor 480, andmay perform a function for temporarily storing data to be input/output.The data to be input/output may include, for example, a file such as avideo, an image, a photo, an audio, or the like. The memory 450 may playa role of storing acquired data in such a manner that data acquired on areal-time basis is stored in a temporary storage device (e.g., a buffer)and data confirmed to be stored is stored in a long-term storage device.

The memory 450 may store one or more programs and data related toexecution of a voice recognition function or the like for processing avoice command by waking up according to a voice call command. Accordingto various embodiments, the memory 450 may store one or more programs,data, or instructions related to allowing the processor 480 to wake upon the basis of detection of a voice call command, to calculate a scorerelated to recognition of the voice call command, to share the scorewith an external device, to decide whether to execute the voice commandin response to the score by interworking with the external device, andto process the voice command on the basis of the decision result.According to various embodiments, the memory 450 may store one or moreprograms, data, or instructions related to allowing the processor 480 tocalculate the score, based at least in part on a distance to a user, anenvironment (e.g., a noise, etc.) of the electronic device 400, alocation (e.g., a spatial or geographical location) of the electronicdevice 400, a domain of the electronic device 400, or a voicerecognition rate.

The memory 450 may include an extended memory (e.g., the external memory234 of FIG. 2 ) or an internal memory (e.g., the internal memory 232 ofFIG. 2 ). The electronic device 400 may operate in association with aweb storage which performs a storage function of the memory 450 on theInternet.

The memory 450 may include one or more application modules (or softwaremodules). For example, a software component may include an operatingsystem software module, a communication software module, a graphicsoftware module, a user interface software module, a Moving PictureExperts Group (MPEG) module, a camera software module, one or moreapplication software modules, or the like. In addition, a module as asoftware component can be expressed as a set of instructions, and thusmay be expressed as an instruction set. The module may also be referredto as a ‘program’. In various embodiments, the memory 450 may includeadditional modules (instructions) in addition to the aforementionedmodule. Optionally, some modules (instructions) may not be used.

The operating system software module may include several softwarecomponents for controlling a general system operation. The control ofthe general system operation may imply, for example, memory managementand control, storage hardware (device) control and management, powercontrol and management, or the like. In addition, the operating systemsoftware module may perform a function for facilitating communicationbetween several hardware components (devices) and software components(programs).

The communication software module may enable communication with otherelectronic devices such as a wearable device, a smart phone, a computer,a server, a television, a monitor, an Internet of Things (IoT) device, amobile terminal, or the like via a communication module (e.g., thewireless communication unit 410, the communication module 220) or aninterface (e.g., the interface 460, the interface 270). In addition, thecommunication software module may consist of a protocol structureassociated with a corresponding communication scheme.

The graphic software module may include several software components forproviding and displaying graphics on a display (e.g., the display 431,the display 260). In various embodiments, the term “graphic” may be usedfor the meaning including a text, a web page, an icon, a digital image,a video, an animation, or the like.

The user interface software module may include several softwarecomponents related to a User Interface (UI). For example, the userinterface software module may include content related to how a state ofthe user interface changes and in which condition the state of the userinterface changes.

The MPEG module may include software components which enable digitalcontent (e.g., video, audio)-related processes and functions (e.g.,content creating, reproduction, distribution, transmission, etc.).

The camera software module may include a camera-related softwarecomponent which enables camera-related processes and functions.

The application module may include a web browser including a renderingengine, an e-mail, an instant message, word processing, keyboardemulation, an address book, a touch list, a widget, a Digital RightManagement (DRM), iris scan, context cognition (or context awareness),voice recognition, a position determining function, a location-basedservice, etc.

The memory 450 may include a computer readable recording mediumrecording a program for executing methods according to variousembodiments in the processor 480.

According to various embodiments, a computer readable recording mediummay include a computer readable recording medium recording a program forexecuting waking up on the basis of detection of a voice call command,calculating a score related to recognition of the voice call command,sharing the score with an external device, deciding whether to executethe voice command corresponding to the score by interworking with theexternal device, and processing the voice command on the basis of thedecision result.

The interface 460 may be constructed identically or similarly, forexample, to the interface 270 of FIG. 2 . The interface 460 may receivedata transmitted from the different electronic device, or may deliversupplied power to each component inside the electronic device 400. Theinterface 460 may allow data inside the electronic device 400 to betransmitted to the different electronic device. For example, awired/wireless headphone port, an external charger port, awired/wireless data port, a memory card port, an audio input/outputport, a video input/output port, an earphone port, or the like may beincluded in the interface 460.

The camera module 470 (e.g., the camera module 291 of FIG. 2 ) isconfigured to support a capturing function of the electronic device 400.The camera module 470 may capture any subject under the control of theprocessor 480, and may deliver captured data (e.g., an image) to thedisplay 431 and the processor 480.

According to various embodiments, the camera module 470 may include, forexample, a first camera (e.g., a color (RGB) camera) for acquiring colorinformation and a second camera (e.g., an InfraRed (IR) camera) foracquiring depth information (e.g., location information and distanceinformation of the subject). The camera module 470 may include an imagesensor. The image sensor may be implemented with a Charged CoupledDevice (CCD) or a Complementary Metal-Oxide Semiconductor (CMOS).According to an embodiment, the first camera may be a front cameraprovided in a front surface of the electronic device 400. According tovarious embodiments, the front camera may be replaced with the secondcamera, and may not be provided in the front surface of the electronicdevice 400. According to various embodiments, the first camera may bedisposed to the front surface of the electronic device 400 together withthe second camera. According to an embodiment, the first camera may be arear camera provided in a rear surface of the electronic device 400.According to an embodiment, the first camera may include both of thefront camera and the rear camera which are provided respectively to thefront surface and the rear surface of the electronic device 400.

The processor 480 (e.g., one or more control circuits including aprocessing circuitry) may provide an overall control to the electronicdevice 400. In various embodiments, the processor 480 may be constructedidentically or similarly, for example, to the processor 120 of FIG. 1 orthe processor 210 of FIG. 2 .

According to various embodiments, the processor 480 may process wakingup on the basis of detection of a voice call command, calculating ascore related to recognition of the voice call command, sharing thescore with an external device, deciding whether to execute the voicecommand corresponding to the score by interworking with the externaldevice, and executing a function corresponding to a voice command on thebasis of the decision result. According to various embodiments, theprocessor 480 may process calculating the score, based at least in parton a distance to a user, an environment of the electronic device 400, alocation of the electronic device 400, a domain of the electronic device400, or a voice recognition rate, transmitting the calculated score toan external device (e.g., a different electronic device, a server),receiving a response determined based on the score from the externaldevice, and determining whether to execute the voice command on thebasis of the received response.

According to various embodiments, the processor 480 may include one ormore processors for controlling the operation of the electronic device400. For example, the processor 480 may include a CommunicationProcessor (CP), an Application Processor (AP), an interface (e.g., aGeneral Purpose Input/Output (GPIO)), an internal memory, and the likeas separate components, or these components may be integrated in one ormore Integrated Circuits (ICs). According to an embodiment, the AP mayperform several functions for the electronic device 400 by executingvarious software programs, and the CP may process and control voicecommunication and data communication. The processor 480 may execute asoftware module (e.g., an instruction set) stored in the memory 450 andthus perform various specific functions corresponding to the module.

In various embodiments, the processor 480 may control an operation of ahardware module such as the audio processor 440, the interface 460, thedisplay 431, the camera module 470, or the like. An operation ofcontrolling (or processing) the processor 480 will be described indetail with reference to drawings described below according to variousembodiments of the disclosure.

According to various embodiments, the processor 480 may be implementedwith one or more processors for controlling the operation of theelectronic device 400 according to various embodiments of the disclosureby executing one or more programs stored in the memory 450. According tovarious embodiments, the processor 480 may be electrically oroperatively coupled to the display 431 and memory 450 of the electronicdevice 400.

The power supplier 490 may supply power required for an operation ofeach component by receiving external power and internal power under thecontrol of the processor 480. In various embodiments, the power supplier490 may supply or turn on/off power to the wireless communication unit410, the display 431, the camera module 470, or the like under thecontrol of the processor 480. According to various embodiments, thepower supplier 490 may include, for example, a battery control circuit.For example, the power supplier 490 may include a battery (e.g., arechargeable battery and/or a solar battery), a battery remaining amountmeasurement circuit (e.g., a fuel gauge), a Power Management IntegratedCircuit (PMIC), a charging circuit, a voltage raising circuit (e.g., abooster circuit), or the like.

FIG. 5A, FIG. 5B, and FIG. 5C illustrate an example of implementing acomponent for processing a voice command in an electronic deviceaccording to various embodiments of the disclosure.

Referring to FIG. 5A, FIG. 5B, and FIG. 5C, FIG. 5A, FIG. 5B, and FIG.5C may show an example of implementing a processor for processing anoperation related to voice command processing in the electronic device400 according to various embodiments. For example, in variousembodiments, the processor 480 may include a first processor 510 relatedto recognition of a voice call command and a second processor 520related to recognition of a voice after wake-up (e.g., a follow-upcommand) and processing of an overall operation related to the voicecommand.

As shown in FIG. 5A, according to various embodiments, the firstprocessor 510 may include a module for recognizing a voice call commandin a state where the electronic device 400 is in a sleep state or anoperating state. According to an embodiment, the first processor 510 mayinclude a wake-up processing unit 530 (e.g., a wake-up engine).

According to various embodiments, the second processor 520 may wake upin response to detection of the wake-up by using the voice call commandof the first processor 510. The second processor 520 may include amodule for performing voice recognition after the wake-up is achieved byusing the first processor 510. According to an embodiment, the secondprocessor 520 may include a follow-up command recognizing unit 540(e.g., a voice recognition engine, an Automatic Speech Recognition (ASR)engine).

According to various embodiments, the first processor 510 may beconfigured in an active state to be able to recognize the voice callcommand, and the second processor 520 may wake up by using the firstprocessor 510, on the basis of detection of the voice call command ofthe first processor 510.

As shown in FIG. 5B, according to various embodiments, the firstprocessor 510 may recognize the voice call command on the basis of thewake-up processing unit 530. The second processor 520 may process anoperation related to the voice command, and the electronic device 400may additionally include a chip (e.g., the follow-up command recognizingunit 540) to recognize a follow-up command.

According to various embodiments, the follow-up command recognizing unit540 may operate to recognize the follow-up command by waking up on thebasis of the voice call command recognition of the first processor 510(e.g., the wake-up processing unit 530). The second processor 520 maywake up by using the first processor 510 or the follow-up commandrecognizing unit 540 to process an operation related to the voicecommand. According to an embodiment, the second processor 520 may wakeup in sequence or in parallel with the wake-up of the follow-up commandrecognizing unit 540. The second processor 520 may have the same orsimilar structure as the processor 210 of FIG. 2 or the processor 480 ofFIG. 4 . According to an embodiment, the second processor 520 may beimplemented with an Application Processor (AP).

For example, the processor 480 may be implemented with the firstprocessor 510 (e.g., the wake-up processing unit 530) for recognizingthe voice call command, a third processor (e.g., the follow-up commandrecognizing unit 540) for recognizing the follow-up command, and thesecond processor 520 for overall processing related to the follow-upcommand. According to an embodiment, the first processor 510 and thefollow-up command recognizing unit 540 may be implemented with anadditional chip (e.g., an wake-up engine, an ASR engine) capable ofprocessing an operation related to voice recognition (e.g., a voice callcommand, a follow-up command), and the second processor 520 may processan operation related to execution of a function corresponding to thevoice command on the basis of recognition of the voice command.

Although not shown, according to various embodiments, the secondprocessor 520 may include a score processing unit (not shown) forcalculating a recognition score for a voice call command, and a sharedprocessing unit (not shown) for sharing the score with an externaldevice (e.g., a different electronic device, a server), a commanddecision unit (not shown) for determining (or deciding) whether toexecute the voice command on the basis of collected information (e.g., ascore, a voice recognition result, etc.).

As shown in FIG. 5C, according to various embodiments, functions of thefirst processor 510 for recognizing the voice call command and thesecond processor 520 for recognizing the follow-up command may beimplemented by one processor. For example, the first processor 510 mayinclude the wake-up processing unit 530 and the follow-up commandrecognizing unit 540. The second processor 520 may process an operationrelated to the voice command after waking up on the basis of the firstprocessor 510.

Although not shown, according to various embodiments, it can beimplemented to recognize the voice call command and the follow-upcommand by using one voice recognition engine (e.g., the first processor510 or the wake-up processing unit 530), instead of additionallyimplementing (or adding) the follow-up command recognizing unit 540 forrecognizing the follow-up command.

According to various embodiments, the aforementioned variousimplementations of the first processor 510 for recognizing the voicecall command and the second processor 520 for recognizing the follow-upcommand after waking up on the basis of recognition of the voice callcommand of the first processor 510 may be divided or integratedaccording to a device type (or feature) of the electronic device 400.For example, according to a device type based on power consumption, thefirst processor 510 and the second processor 520 may be implementedindependently or may be implemented in a form of one chip.

According to an embodiment, if the electronic device 400 is a device towhich power is always connected such as a refrigerator, an airconditioner, a television, or the like, it can be implemented by asingle processor without having to add a separate chip (e.g., the secondprocessor 520) for processing the follow-up command.

According to an embodiment, if the electronic device 400 is a devicewhich uses a high-capacity battery such as an automotive infotainmentdevice, it can be implemented by a single processor without having toadd a separate chip for processing the follow-up command.

According to an embodiment, if the electronic device 400 is a devicewhich uses a low-power battery such as a smart phone, a watch, aBluetooth speaker, or the like, a low-power chip (e.g., the secondprocessor 520) capable of recognizing a command may be added in additionto the first processor 510. Alternatively, according to variousembodiments, the electronic device 400 may be designed such that boththe voice call command and the command can be recognized while using onelow-power chip (e.g., the first processor 510 or the second processor520).

FIG. 6 illustrates an example of an electronic device according tovarious embodiments of the disclosure.

Referring to FIG. 6 , in various embodiments, the electronic device 400may include various devices having a component related to voicerecognition and a microphone. In various embodiments, the electronicdevice 400 may include various types of devices capable of executing afunction related to a voice command by receiving a voice input based onuser's utterance through the microphone. For example, the electronicdevice 400 may include a smartphone 610, a computer 620 (e.g., apersonal computer, a notebook, etc.), a Bluetooth speaker 630, an IoTdevice 640, a television 650, a refrigerator 660, or the like. Althoughnot shown, the electronic device 400 according to various embodimentsmay be implemented by using various devices such as a tablet PC, anautomotive infotainment device, a washing machine, an air conditioner,or the like, in addition to the aforementioned structure.

According to various embodiments, the plurality of electronic devices610, 620, 630, 640, 650, and 660 may be registered (or set) with thesame or different voice call command. In various embodiments, a casewhere the same voice call command is set in the plurality of electronicdevices 610, 620, 630, 640, 650, and 660 will be described for example.According to an embodiment, the plurality of electronic devices 610,620, 630, 640, 650, and 660 may be set with a specific wakeup word suchas “hi”, “hello”, “hi, ABC”, or the like. In various embodiments, “ABC”may represent, for example, a name of an electronic device (or an agentof an electronic device (or an Artificial Intelligence (AI)). Forexample, it may represent a name given such as galaxy or the like.

According to various embodiments, the electronic devices 610, 620, 630,640, 650, and 660 may be set to a state capable of recognizing a user'svoice call command. For example, the electronic devices 610, 620, 630,640, 650, and 660 may be activated in a state of waiting for receptionof the voice command by waking up substantially simultaneously on thebasis of the user's voice call command. According to an embodiment, theelectronic devices 610, 620, 630, 640, 650, and 660 may be in a state ofreceiving an audio input via a microphone, and may detect a voice callcommand by using an wake-up engine (e.g., the first processor 510) fromthe audio input. A voice recognition engine (e.g., the second processor520) may wake up by using the wake-up engine. The voice recognitionengine may receive a voice signal (or a voice command) for utterance ofa user 600 according to the wake-up, and may perform voice recognition.

According to various embodiments, the electronic devices 610, 620, 630,640, 650, and 660 may be set as one group on the basis of an account ofthe user 600. For example, as exemplified in Table 1 below, theelectronic devices 610, 620, 630, 640, 650, and 660 may be grouped withthe same account (e.g., samsung@samsung.com). Information related to theaccount and grouping of the electronic devices 610, 620, 630, 640, 650and 660 may be registered in advance in the electronic devices 610, 620,630, 640, 650, and 660 and/or a server (e.g., a Samsung server, a cloudserver, etc.) as information related to the voice command processing.For example, the related information (hereinafter, service information)may be registered by including an account, device information(category), a score, a domain, or the like.

TABLE 1 Name (nickname, device Domain Account Category identifier) Score(command) First account Phone Galaxy XXX phone message music . . . TVBordeaux XXX channel volume music . . . Refrigerator Zipel XXX recipetemperature control weather . . . . . . . . . . . . . . . Second . . . .. . . . . . . . account

According to various embodiments, the electronic devices 610, 620, 630,640, 650, and 660 may be associated with the server and set to a standbystate capable of receiving a response from the server. According tovarious embodiments, the electronic devices 610, 620, 630, 640, 650, and660 may be grouped with the same account, coupled with one another, andset to the standby state capable of receiving a response from otherelectronic devices.

According to various embodiments, the electronic devices 610, 620, 630,640, 650, and 660 may be identified based on unique names (e.g., anickname, a device identifier) of the respective devices in the sameaccount.

According to various embodiments, the electronic devices 610, 620, 630,640, 650, and 660 may each include a domain list of domains that can beprocessed by themselves. For example, the electronic device 610, 620,630, 640, 650, and 660 may be set with a domain list on the basis ofexecutable functions. In various embodiments, a domain may represent afunction or task which can be executed by the electronic devices 610,620, 630, 640, 650, and 660 by using the voice command. According to anembodiment, in case of a phone, a domain of a phone, message, music, orthe like may be set. According to an embodiment, in case of TV, a domainof a channel, volume, music, or the like, may be set. According to anembodiment, in case of a refrigerator, a domain of a recipe, temperaturecontrol, weather, or the like may be set.

According to various embodiments, the electronic devices 610, 620, 630,640, 650, and 660 may calculate a recognition score for the voice callcommand in sequence or in parallel at a time of recognizing the voicecall command. According to an embodiment, the electronic devices 610,620, 630, 640, 650, and 660 may interwork with the server to transferthe recognized score to the server in a method of processing a voicecommand. According to an embodiment, the electronic devices 610, 620,630, 640, 650, and 660 may transfer the recognized score to a differentelectronic device in the same account and share the score with eachother, in a method in which the voice command is processed based on theelectronic device without interworking with the server.

In various embodiments, the score represents a recognition rate by whichthe electronic devices 610, 620, 630, 640, 650, and 660 recognize thevoice call command of the user 600, and the score may be calculated byconsidering various situations. For example, it may be assumed that thescore is calculated as exemplified in Equation (1) below.

$\begin{matrix}\left\{ \begin{matrix}{{Success},} & {{{if}p\left( O \middle| \lambda \right)} > {Th}} \\{{fail},} & {otherwise}\end{matrix} \right. & \left\lbrack {{Equation}1} \right\rbrack\end{matrix}$

Referring to Equation 1, “O” may denote an observation feature sequence,“λ” may denote a recognition model or a recognition word model, “Th” maydenote a threshold, and “p(O|λ)” may denote a recognition score.According to an embodiment, for “λ” denoting the recognition model, aHidden Markov Model (HMM), an Artificial Neural Network (ANN), or thelike may be used.

According to various embodiments, the score may be calculated based on avariety of information (e.g., distance information, surroundingenvironment information (e.g., noise information), etc.) among theelectronic devices 610, 620, 630, 640, 650, and 660. This will beexplained with reference to drawings described below.

According to various embodiments, the electronic devices 610, 620, 630,640, 650, and 660 may be grouped for each account of the user. Forexample, as shown in the example of Table 1, a phone, a TV, arefrigerator, and the like may be configured as one group by a firstaccount, and other electronic devices may be configured as another groupby a second account. According to various embodiments, the user mayconfigure a voice call command different for each account to controlelectronic devices on a group basis.

According to various embodiments, each of the electronic devices 610,620, 630, 640, 650, and 660 or the server may manage or use a devicename (a nickname) of devices grouped for each account, an availabledomain list, score information for voice call command recognition, orthe like during a session of the electronic devices associated with thesame account. This will be explained in detail with reference todrawings described below.

In various embodiments, the electronic devices 610, 620, 630, 640, 650,and 660 may be coupled with one another. For example, in variousembodiments, the electronic devices 610, 620, 630, 640, 650, and 660 maybe coupled with other electronic devices on the basis of wirelesscommunication (e.g., Bluetooth, Bluetooth Low Energy (BLE), WiFi, etc.).In various embodiments, the electronic devices 610, 620, 630, 640, 650,and 660 may wirelessly communicate with different electronic devices,and may exchange a variety of information (e.g., score information)related to the voice command processing with the different electronicdevices. According to an embodiment, the electronic device 400 maytransmit a score calculated by itself to a different electronic device,and may receive a score calculated by the different electronic devicefrom the different electronic device. According to various embodiments,various examples related to processing of the voice command by theelectronic device 400 to exchange the score will be described in detailwith reference to the accompanying drawings.

According to various embodiments, the electronic devices 610, 620, 630,640, 650, and 660 may be directly coupled through wirelesscommunication. According to various embodiments, the electronic devices610, 620, 630, 640, 650, and 660 may be coupled through an Access Point(AP) which is commonly accessed.

As described above, an electronic device according to variousembodiments of the disclosure includes a microphone, a memory, and aprocessor operatively coupled to the microphone and the memory. Theprocessor may be configured to wake up on the basis of detection of avoice call command, calculate a score related to recognition of thevoice call command, share the score with an external device, decidewhether to execute a voice command on the basis of the score, andprocess the voice command on the basis of the decision result.

According to various embodiments, the processor may be configured toinclude a first processor configured to recognize the voice call commandand a second processor configured to wake up on the basis of detectionof the voice call command of the first processor and to process voicerecognition and an operation related to the voice command.

According to various embodiments, the processor may be configured tocalculate the score, based at least in part on a distance to a user, anenvironment of the electronic device, a location of the electronicdevice, a domain of the electronic device, or a voice recognition rate.

According to various embodiments, the processor may be configured totransmit the calculated score to the external device, receive a responsedetermined based on the score from the external device, and determinewhether to process the voice command on the basis of the receivedresponse.

According to various embodiments, the processor may be configured toprocess the voice command if the response from the external device is afirst response for instructing to process the voice command, and totransition to a standby state if the response from the external deviceis a second response for instructing to transition to the standby state.

According to various embodiments, the processor may be configured tooutput a related feedback if the response from the external device is afeedback response for requesting a user to select a device, and toacquire a device designation command of the user for the feedback.

According to various embodiments, the processor may be configured todecide a score of the electronic device and a score acquired by theexternal device, and if the score of the electronic device is high,determine the device as a target device for processing the voicecommand.

According to various embodiments, the processor may be configured todetermine whether to process the voice command on the basis of a devicedesignation command or a domain command.

According to various embodiments, the processor may be configured todetermine a feedback target device on the basis of the score if aplurality of electronic devices include a related domain among devicesgrouped with the same account on the basis of the domain command,acquire device selection performed by a user on the basis of thedetermined feedback target device, and determine an electronic devicecorresponding to the device selection performed by the user as a targetdevice for executing a voice command.

According to various embodiments, the processor may be configured toacquire the determination of the feedback target device and thedetermination of the target device from the external device.

According to various embodiments, the external device may be configuredto determine a target device for executing the voice command, based atleast in part on a score of electronic devices grouped with the sameaccount, a device designation command, or a domain command, transmit afirst response for instructing to process the voice command to thedetermined target device, and transmit a second response for instructinga different electronic device other than the target device among theelectronic devices to transition to a standby state. The processor maybe configured to process the voice command on the basis of receiving ofthe first response from the external device, or transition to thestandby state on the basis of receiving of the second response.

According to various embodiments, the voice command may be determined tobe processed by any one of a plurality of electronic devices which wakeup on the basis of the voice call command and which are grouped with thesame account.

FIG. 7 , FIG. 8 , and FIG. 9 illustrate an example of an operation ofprocessing a voice command in a system according to various embodimentsof the disclosure.

As shown in FIG. 7 and FIG. 8 , a first electronic device 710 (e.g., aphone), a second electronic device 720 (e.g., a TV), and a thirdelectronic device 730 (e.g., a refrigerator) may be grouped based on aspecific account (e.g., samsung@samsung.com), and the same voice callcommand (e.g., hi, galaxy) may be registered. According to anembodiment, it may be assumed that the first electronic device 710includes a domain list of a phone, a message, music, or the like, thesecond electronic device 720 includes a domain list of a channel, avolume, music, or the like, and the third electronic device 730 includesa domain list of a recipe, temperature control, weather, or the like.According to various embodiments, the first electronic device 710, thesecond electronic device 720, and the third electronic device 730 mayrespectively include microphones 715, 725, and 735, and may receive anaudio input on the basis of utterance of a user 700 through therespective microphones 715, 725, and 735.

Referring to FIG. 7 , the first electronic device 710, the secondelectronic device 720, and the third electronic device 730 may becoupled to a server 750 on the basis of the same account (e.g.,samsung@samsung.com), and may be in a standby state capable of receivinga response from the server 750. According to various embodiments, theserver 750 may include one or more control circuits for processing anoperation related to voice command processing depending on variousembodiments. According to an embodiment, the server 750 may include aprocessor corresponding to the processor 480 of the electronic device400, and may include a voice recognition processing unit for allowingthe electronic device to process voice recognition and related commands.According to an embodiment, the server 750 may include a database forstoring a variety of information (e.g., account information, deviceinformation, domain list, etc.) for managing the electronic devices.

In a state shown in FIG. 7 , the user 700 may utter a specific commandto process a voice command in a specific electronic device. According toan embodiment, the user 700 may utter a configured voice call commandfor waking up the electronic device. According to an embodiment, theuser 700 may utter the voice call command and a follow-up command (e.g.,a command related to function execution). For example, the example ofFIG. 8 may show a case where the user utters the voice call command andthe follow-up command together.

Referring to FIG. 8 , in operation 801, the user 700 may utter a command(e.g., a voice call command+a follow-up command) for wake-up andfunction execution in a corresponding electronic device to initiate afunction (e.g., music playback) based on a voice command by a specificelectronic device (e.g., a phone). According to an embodiment, the usermay utter such as “Hi, Galaxy, play music.” Herein, “Hi, Galaxy” maycorrespond to the voice call command, and “play music” may correspond tothe follow-up command.

According to various embodiments, a plurality of electronic devices (notshown) may receive an audio input based on utterance of the user 700through the microphone (e.g., 715, 725, or 735). According to variousembodiments, among the plurality of electronic devices, the first,second, and third electronic devices 710, 720, and 730 in which “Hi,Galaxy” is registered as the voice call command may wake up in responseto the voice call command (e.g., Hi, Galaxy) of the user 700.

In operations 803, 805, and 807, the first, second, and third electronicdevices 701, 720, and 730 may calculate a recognition score for thevoice call command of the user 700 on the basis of the wake-up. Theexample of FIG. 8 may show a case where the first electronic device 710calculates a score “400”, the second electronic device 720 calculates ascore “700”, and the third electronic device 730 calculates a score“200”. According to various embodiments, the score may be calculated inthe same manner as explained with reference to the aforementionedEquation 1. In addition, the score may be calculated by considering avariety of information as shown in the example of FIG. 9 describedbelow.

FIG. 9 illustrates an example of information considered when a score iscalculated in an electronic device according to various embodiments.

Referring to FIG. 9 , the electronic device 400 may calculate a score byconsidering a distance 910 to the user 700, an environment 920 (e.g., anoise, etc.) of the electronic device 400, a location 930 (e.g., aspatial or geographical location) of the electronic device 400, a domain940 of the electronic device 400, a recognition rate 950, or the like.

According to various embodiments, the first electronic device 710, thesecond electronic device 720, and the third electronic device 730 mayupdate information (e.g., service information) related to voice commandprocessing exemplified in Table 1 above on the basis of the scorecalculation as shown in the example of Table 2 below. For example, thefirst electronic device 710, the second electronic device 720, and thethird electronic device 730 may register the calculated scoreinformation in a score item.

TABLE 2 Name (nickname, device Domain Account Category identifier) Score(command) First account Phone Galaxy 400 phone message music . . . TVBordeaux 700 channel volume music . . . Refrigerator Zipel 200 recipetemperature control weather . . . . . . . . . . . . . . . Second . . . .. . . . . . . . account

In operations 809, 811, and 813, the first electronic device 710, thesecond electronic device 720, and the third electronic device 730 maytransmit score information on the calculated score to the server 750.According to various embodiments, the first electronic device 710, thesecond electronic device 720, and the third electronic device 730 maytransmit service information in which the score is input as shown in theexample of Table 2 to the server 750. In the example of Table 2, each ofthe first electronic device 710, the second electronic device 720, andthe third electronic device 730 may transmit only scores correspondingthereto as shown in the example of Table 3 below, and a form shown inTable 2 may be collected and managed by the server 750.

TABLE 3 Name (nickname, device Domain Account Category identifier) Score(command) First account Phone Galaxy 400 phone message music . . .

According to various embodiments, the first electronic device 710, thesecond electronic device 720, and the third electronic device 730 mayperform transmission to the server 750 by including only a name fordevice identification and a calculated score.

Upon receiving service information from the first electronic device 710,the second electronic device 720, and the third electronic device 730,the server 750 may decide a target device for executing a voice commandon the basis of the score of the first electronic device 710, the secondelectronic device 720, and the third electronic device 730. For example,the server 750 may decide (or determine) an electronic device (e.g., thesecond electronic device 720) having a highest score, and may determinethe electronic device as the target device for executing the voicecommand. The server 750 may determine that an electronic device (e.g.,the first electronic device 710, the third electronic device 730) otherthan the decided electronic device (e.g., the second electronic device720) is to wait.

According to various embodiments, the server 750 may identify devicesgrouped with the same account, on the basis of service information foreach device as shown in the example of Table 2. According to anembodiment, the server 750 may pre-register and manage serviceinformation as shown in the example of Table 1, and may change theservice information as shown in Table 2 in response to reception ofscore information. According to an embodiment, upon determining anexecution target device, the server 750 may initiate the scoreinformation in the service information.

In operations 815, 817, and 819, the server 750 may transmit a response(e.g., a first response, a second response) to the first electronicdevice 710, the second electronic device 720, and the third electronicdevice 730 on the basis of a decision result. According to anembodiment, the server 750 may transmit the first response to anelectronic device (e.g., the second electronic device 720) determined asthe execution target device to execute a voice command. According to anembodiment, the server 750 may transmit the second response to anelectronic device (e.g., the first electronic device 710, the thirdelectronic device 730) other than the execution target device totransition to a standby state.

According to various embodiments, the first electronic device 710, thesecond electronic device 720, and the third electronic device 730 mayreceive a response from the server 750, and may perform a correspondingoperation on the basis of the received response. According to anembodiment, the first electronic device 710 and the third electronicdevice 730 may transition to the standby state in response to receptionof the second response. According to an embodiment, the secondelectronic device 720 may perform an operation related to a voicecommand based on user's utterance, in response to reception of the firstresponse.

Meanwhile, according to various embodiments, if a follow-up command isincluded as shown in the example of FIG. 8 , the first electronic device710, the second electronic device 720, and the third electronic device730 may perform transmission to the server 750 by including serviceinformation and the follow-up command.

Upon receiving the service information and the follow-up command fromthe first electronic device 710, the second electronic device 720, andthe third electronic device 730, the server 750 may decide a devicecapable of executing a command corresponding to the follow-up command.According to an embodiment, if the follow-up command is included, theserver 750 may determine a device for performing an operation withreference to device information (e.g., a device name or a nickname, adomain list) among the first electronic device 710, second electronicdevice 720, and third electronic device 730 of the same account.

According to various embodiments, the server 750 may determine theexecution target device by preferentially considering whether a commandis executable based on the follow-up command with a higher priority thanthe score of first electronic device 710, second electronic device 720,and third electronic device 730. For example, even if a specificelectronic device is determined based on the score, the server 750 maydetermine a different electronic device capable of executing a commandas a target device. For another example, if the follow-up command isincluded, the server 750 may determine an execution target device bypreferentially identifying whether the command is executable based onthe follow-up command.

According to various embodiments, if the follow-up command includes adevice designation command, the server 750 may identify a device basedon a designation command among the first, second, and third electronicdevices 710, 720, and 730 grouped with the same account. For example, onthe basis of device information (e.g., a device name (or a nickname), adevice identifier, etc.) of devices grouped with the same account, theserver 750 may identify an electronic device having corresponding deviceinformation, and may determine the identified electronic device as atarget device for executing a voice command.

According to various embodiments, the server 750 may identify a domainlist and thus identify one or more devices capable of executing acommand among the first, second, and third electronic devices 710, 720,and 730 grouped with the same account.

According to an embodiment, if it is decided that there is one devicecapable of executing the command, the server 750 may transmit a firstresponse for executing a voice command to a corresponding device capableof executing the command among the first, second, and third electronicdevices 710, 720, and 730 grouped with the same account. The server 750may transmit a second response for transitioning to the standby state toa different electronic device not capable of executing the command.

According to an embodiment, if a plurality of (e.g., at least two)devices are capable of executing the command, the server 750 maydetermine a feedback target device on the basis of a score ofcorresponding electronic devices. According to an embodiment, the server750 may determine an electronic device having a highest score among theplurality of electronic devices capable of executing the command as adevice for feedback of the user 700. For example, the server 750 mayprovide the feedback to the user 700 to determine (or select) anexecution target device for executing a voice command by using Text ToSpeech (TTS) or other methods or the like. According to variousembodiments, the server 750 may transmit to a feedback target device(e.g., the second electronic device 720) a third response (e.g., afeedback response) to request for feedback, and may transmit to adifferent electronic device (e.g., the first electronic device 710, thethird electronic device 730) a different response to wait for aresponse. According to an embodiment, the server 750 may transmit to thedifferent electronic device a second response to transition to thestandby state.

For example, referring to FIG. 8 , it may be assumed that a domainassociated with a follow-up command, i.e., a command of “play music”, is“music”. As shown in FIG. 8 , it may be assumed that a domain “music” isincluded in a domain list in the first electronic device 710 and thesecond electronic device 720. In this case, the server 750 may determinethe second electronic device 720 having a higher score between the firstelectronic device 710 and the second electronic device 720 as a feedbacktarget device. Upon determining the feedback target device, as shown inoperation 821, the server 750 may transmit a command (e.g., a feedbackcommand) to the second electronic device 720 determined as the feedbacktarget device to request for designation (or selection) of the targetdevice for executing the command. According to various embodiments, whenthe feedback command is provided, the server 750 may provide informationon a device (e.g., the first electronic device 710, the secondelectronic device 720) that can be selected (or selectable) according tothe feedback command.

Upon receiving the feedback command from the server 750, the secondelectronic device 720 may provide a feedback for requesting the user todesignate a device in response to the feedback command. According to anembodiment, as shown in operation 823, the second electronic device 720may request the user to select a device for executing a music playbackcommand such as “On what device do you want to play music? You can playmusic on your phone and TV.”. According to various embodiments, thesecond electronic device 720 may guide information on selectable deviceson the basis of device information received from the server 750.

The user 700 may utter a voice for designating (selecting) a specificdevice in response to a device selection feedback of the secondelectronic device 720. According to an embodiment, as shown in operation825, the user 700 may utter a voice for designating (selecting) thefirst electronic device 710 such as “phone” or “play music on phone” orthe like. Upon detecting a voice input based on the utterance of theuser 700, the second electronic device 720 may transmit the input voiceinformation to the server 750.

Upon receiving a response (e.g., voice information) corresponding to thefeedback command from the second electronic device 720, the server 750may decide an execution target device on the basis of the receivedresponse. According to an embodiment, the server 750 may analyze thereceived response to determine the first electronic device 710 as afinal execution target device on the basis of the received responsebetween the first and second electronic devices 710 and 720 capable ofexecuting a function (e.g., music playback) based on the voice command.

Upon determining the execution target device, the server 750 maytransmit a first response to the determined execution target device toinstruct to execute the voice command (e.g., music playback). Accordingto an embodiment, as shown in operation 829, the server 750 may transmitthe first response to the first electronic device 710. According tovarious embodiments, the server 750 may transmit a second response to anelectronic device (e.g., the second electronic device 720, the thirdelectronic device 730) other than the execution target device totransition to the standby state. According to various embodiments, thesecond response may be transmitted in advance and thus the thirdelectronic device 730 may be in the standby state. Therefore, the server750 may transmit the second response to the second electronic device720, and may not perform response transmission to the third electronicdevice 730.

The first electronic device 710 and the second electronic device 720 mayreceive a response from the server 750, and may perform a correspondingoperation on the basis of the received response. According to anembodiment, the first electronic device 710 may perform a function(e.g., music playback) corresponding to the voice command as shown inoperation 831 in response to reception of the first response. Accordingto an embodiment, the second electronic device 720 may transition to thestandby state in response to reception of the second response.

FIG. 10 illustrates an operation of processing a voice command in asystem according to various embodiments of the disclosure.

As shown in FIG. 10 , a case shown in FIG. 10 may be as case where afirst electronic device 1010, a second electronic device 1020, and athird electronic device 1030 are grouped based on a specific account,and the same voice call command is registered. In addition, FIG. 10 mayshow an example of processing a voice command by interworking with aserver 1040.

Referring to FIG. 10 , in operation 1001, a user may utter a voice callcommand so that a voice command is executed in a specific electronicdevice. According to various embodiments, the user may utter the voicecall command and a follow-up command, and may separately utter thefollow-up command after waking up the electronic device on the basis ofthe voice call command.

In operation 1003, the first electronic device 1010, the secondelectronic device 1020, and the third electronic device 1030 may wake upon the basis of the voice call command. According to variousembodiments, the first electronic device 1010, the second electronicdevice 1020, and the third electronic device 1030 may simultaneouslywake up within a specific time range in sequence or in parallel at atime of recognizing the voice call command. According to variousembodiments, the first electronic device 1010, the second electronicdevice 1020, and the third electronic device 1030 may receive andrecognize the voice call command of the user through a microphone.

In operation 1005, the first electronic device 1010, the secondelectronic device 1020, and the third electronic device 1030 maycalculate a recognition score of the voice call command. In variousembodiments, the first electronic device 1010, the second electronicdevice 1020, and the third electronic device 1030 may calculate arecognition score for the voice call command of the user on the basis ofthe wake-up caused by the voice call command.

In operations 1007, 1009, and 1011, the first electronic device 1010,the second electronic device 1020, and the third electronic device 1030may transmit the calculated score to the server 1040. According tovarious embodiments, the first electronic device 1010, the secondelectronic device 1020, and the third electronic device 1030 may includeand provide device information for identifying a device to the server1040. According to an embodiment, the first electronic device 1010, thesecond electronic device 1020, and the third electronic device 1030 mayprovide service information including the score and device informationto the server 1040. According to various embodiments, if another voicecommand (e.g., a follow-up command) is included in voice recognition,the first electronic device 1010, the second electronic device 1020, andthe third electronic device 1030 may transmit service information to theserver 1040 by including the voice command.

In operation 1013, upon receiving the service information from the firstelectronic device 1010, the second electronic device 1020, and the thirdelectronic device 1030, the server 1040 may determine a target devicefor executing the voice command on the basis of the score of the firstelectronic device 1010, the second electronic device 1020, and the thirdelectronic device 1030. For example, the server 1040 may decide (ordetermine) an electronic device having a highest score, and maydetermine the electronic device as the target device for executing thevoice command. According to various embodiments, the server 1040 maydetermine that an electronic device other than the electronic devicedetermined as the target device is to wait. According to an embodiment,FIG. 10 may show a case where the first electronic device 1010 has ahighest score, and the server 1040 determines the first electronicdevice 1010 as the target device. According to various embodiments, theserver 1040 may pre-store a variety of information related to the firstelectronic device 1010, the second electronic device 1020, and the thirdelectronic device 1030, and may determine the target device on the basisof the stored variety of information. This will be described in detailwith reference to the accompanying drawings.

In operations 1015, 1017, and 1019, the server 1040 may transmit aresponse (e.g., a first response, a second response) to the firstelectronic device 1010, the second electronic device 1020, and the thirdelectronic device 1030 on the basis of a decision result. According toan embodiment, in operation 1015, the server 1040 may transmit a firstresponse to the first electronic device 1010 determined as the targetdevice to execute the voice command among the first electronic device1010, the second electronic device 1020, and the third electronic device1030. According to an embodiment, in operations 1017 and 1019, theserver 1040 may transmit a second response to the second electronicdevice 1020 and third electronic device 1030, which are not selected asthe target device, to transition to a standby state.

In operations 1021, 1023, and 1025, the first electronic device 1010,the second electronic device 1020, and the third electronic device 1030may receive a response from the server 1040, and may perform acorresponding operation on the basis of the received response. Accordingto an embodiment, in operation 1021, the first electronic device 1010may operate to process the voice command in response to reception of thefirst response. For example, the first electronic device 1010 mayoperate to perform a related service based on voice recognition (e.g.,processing of the voice command and executing of a function related tothe voice command). According to an embodiment, in operations 1023 and1025, the second electronic device 1020 and the third electronic device1030 may operate to transition to the standby state in response toreception of the second response.

FIG. 11 is a flowchart illustrating an operation of processing a voicecommand in an electronic device according to various embodiment of thedisclosure.

Referring to FIG. 11 , in operation 1101, the processor 480 of theelectronic device 400 may detect a voice call command. According to anembodiment, the electronic device 400 may recognize the voice callcommand through a microphone on the basis of a first processor (e.g., awake-up processing unit, a wake-up engine) for recognizing the voicecall command.

In operation 1103, the processor 480 may wake up in response todetection of the voice call command. According to an embodiment, theelectronic device 400 may wake up a second processor (e.g., a follow-upcommand recognition unit, a voice recognition engine, a service controlcircuit), on the basis of voice call command detection of the firstprocessor.

In operation 1105, the processor 480 may calculate a recognition scoreof the voice call command. According to an embodiment, the processor 480may calculate a score by considering various situations, as explainedwith reference to Equation 1 and FIG. 9 .

In operation 1107, the processor 480 may transmit to the server arequest message for identifying whether to execute the voice command.According to an embodiment, upon calculating the score, the processor480 may transmit service information including at least part of thecalculated score information, device information, or a follow-up commandto the server by using a configured communication interface (e.g., thewireless communication unit 410). According to various embodiments, theservice information may include information for requesting the server toidentify whether to execute the voice command.

In operation 1109, the processor 480 may receive from the server aresponse corresponding to a request for identifying whether to executean operation.

In operation 1111, the processor 480 may decide whether the receivedresponse corresponds to a first response for instructing to initiate thevoice command or a second response for instructing to transition to astandby state.

If it is decided in operation 1111 that the received response is thefirst response (YES in operation 1111), in operation 1113, the processor480 may process the voice command. For example, the processor 480 mayexecute a voice recognition operation for user's utterance, and mayprocess an operation related to execution of the related function.

If it is decided in operation 1111 that the received response is thesecond response (NO in operation 1111), in operation 1115, the processor480 may transition to the standby state to wait for the voice callcommand. According to an embodiment, the processor 480 may allow theelectronic device 400 to transition to a sleep state. For example, theelectronic device 400 may allow a second processor to transition to thesleep state. The electronic device 400 in the sleep state may wait forthe voice call command on the basis of the first processor, and may notexecute the voice recognition operation for user's utterance caused bythe second processor.

FIG. 12 is a flowchart illustrating an operation of processing a voicecommand in a server according to various embodiment of the disclosure.

Referring to FIG. 12 , in operation 1201, the server 750 (or a controlcircuit of the server 750) may receive a message (e.g., serviceinformation) to request for identifying whether to execute a voicecommand from at least one electronic device. According to variousembodiments, the service information may include score information,device information, a follow-up command, or the like.

In operation 1203, the server 750 may decide a score for each device.According to an embodiment, upon receiving the service information fromthe first electronic device, the second electronic device, and the thirdelectronic device, the server 750 may compare and decide a first scoreof the first electronic device, a second score of the second electronicdevice, and a third score of the third electronic device. According tovarious embodiments, the first electronic device, the second electronicdevice, and the third electronic device may be devices grouped with thesame account.

In operation 1205, the server 750 may determine a target device forexecuting the voice command on the basis of the score. According to anembodiment, the server 750 may decide an electronic device having ahighest score, and may determine the electronic device as the targetdevice for executing the voice command. The server 750 may determinethat an electronic device other than the electronic device determined asthe target device is to wait.

In operation 1207, the server 750 may transmit a first response to atarget device determined to execute a voice command. According tovarious embodiments, the first response may include a response forinstructing the target device to execute the voice command.

In operation 1209, the server 750 may transmit a second response to adifferent device determined not to execute the voice command, forexample, a different device determined to transition to a standby state.According to various embodiments, the second response may include aresponse for instructing the different device to transition to thestandby state (or sleep state) for waiting for a voice call command.

In various embodiments, the order of operations 1207 and 1209 is shownfor convenience of explanation, and is not limited to the order of FIG.12 . For example, operations 1207 and 1209 may be performed in sequence,in parallel, or in a reverse order.

FIG. 13 is a flowchart illustrating an operation of processing a voicecommand in a server according to various embodiments of the disclosure.

Referring to FIG. 13 , in operation 1301, the server 750 (or a controlcircuit of the server 750) may receive a request related to a voicecommand from an external device. According to various embodiments, theexternal device may include a plurality of electronic devices.

In operation 1303, the server 750 may decide whether the request fromthe external device is a first request for identifying whether toexecute a voice command or a second request for requesting for a result(or command) on voice recognition. According to an embodiment, theserver 750 may identify a request category on the basis of parsing of atype (or format) (e.g., service information) of a request received fromthe external device, header information of a request message, wake-uprelated information included in the request message, or the like.

If it is decided as the second request in operation 1303 (NO inoperation 1303), in operation 1305, the server 750 may recognize a voicecommand and transmit a response thereon to a corresponding externaldevice.

If it is decided as the first request in operation 1303 (YES inoperation 1303), in operation 1307 m the server 750 may decide whether adetermination condition to be considered to determine a target device isincluded in the first request (e.g., service information). According tovarious embodiments, the determination condition may include, forexample, at least one of the number of devices which have transmittedthe request, a device designation command (e.g., “phone”, etc.), adomain command (e.g., “music playback”, etc.), or the like. For example,when at least one follow-up command such as the device designationcommand and/or the domain command or the like is included in the firstrequest, the server 750 may decide that the determination condition isincluded.

If it is decided in operation 1307 that the first request includes thedetermination condition (YES in operation 1307), in operation 1309, theserver 750 may determine a target device on the basis of at least onecondition for determining the target device for executing the voicecommand. A specific example for this will be described in detail withreference to FIG. 14 .

If it is decided in operation 1307 that the first request does notinclude the determination condition (NO in operation 1307), in operation1311, the server 750 may decide a score for each device on the basis ofdevices grouped with the same account.

In operation 1313, the server 750 may determine a target device forexecuting the voice command on the basis of the score. For example, asdescribed above, the server 750 may determine an electronic devicehaving a highest score as the target device among the electronic devicesof the same account.

In operation 1315, the server 750 may transmit a different response toeach of the target device and a different device. According to anembodiment, the server 750 may transmit a first response to instruct thetarget device to process a voice command and a second response toinstruct the different device other than the target device to transitionto a standby state.

FIG. 14 is a flowchart illustrating an example of determining a targetdevice for executing a voice command in a server according to variousembodiments of the disclosure.

Referring to FIG. 14 , in operation 1401, the server 750 (or a controlcircuit of the server 750) may analyze a request received from anexternal device. According to an embodiment, upon receiving a request(e.g., service information) for identifying whether to process the voicecommand from the external device, the server 750 may decide adetermination condition to be considered to determine whether therequest is received from one external device, whether a devicedesignation command (e.g., “phone”, etc.) is included, whether a domaincommand (e.g., “music playback”, etc.) is included, or the like.According to various embodiments, the request may include one or moredetermination conditions.

In operation 1411, the server 750 may decide whether a request isreceived from one external device. For example, the server 750 maydecide whether there is a request from a different device belonging tothe same account as that of a corresponding external device.

If it is decided in operation 1411 that the request is received from oneexternal device (YES in operation 1411), in operation 1413, the server750 may determine a corresponding external device as a target device forexecuting a voice command.

If it is decided in operation 1411 that the request is received from theplurality of external devices of the same account (NO in operation1411), in operation 1421, the server 750 may decide whether a devicedesignated command is included.

If it is decided in operation 1421 that the request includes the devicedesignation command (YES in operation 1421), the server 750 may identifya designated device among the devices grouped with the same account. Forexample, the server 750 may identify a device based on the devicedesignation command among the devices grouped with the same account.According to an embodiment, on the basis of device information (e.g., adevice name (or a nickname), a device identifier, etc.) of the devicesgrouped with the same account, the server 750 may identify an electronicdevice having corresponding device information.

In operation 1425, the server 750 may determine the identified device(e.g., a device designated by a user) as the target device for executingthe voice command.

If it is decided in operation 1421 that the device designation commandis not included (NO in operation 1421), in operation 1431, the server750 may decide whether the request includes a domain command.

If it is decided in operation 1431 that the request does not include thedomain command (NO in operation 1431), in operation 1451, the server 750may determine the target device for executing the voice command on thebasis of scores of devices grouped with the same account. For example,an external device having a highest score may be determined as thetarget device on the basis of the score.

If it is decided in operation 1431 that the request includes the domaincommand (YES in operation 1431), in operation 1433, the server 750 mayanalyze the domain list.

In operation 1435, the server 750 may identify one or more devicescapable of executing a command (function) based on a domain commandamong devices grouped with the same account, on the basis of an analysisresult of a domain list.

In operation 1437, the server 750 may decide whether there are aplurality of devices capable of executing the command. For example, itmay be decided whether there are devices having the same domain amongthe devices grouped with the same account.

If it is decided in operation 1437 that there is one device capable ofexecuting the command (NO in operation 1437), in operation 1439, theserver 750 may determine a corresponding device capable of executing acommand as the target device for executing the voice command among thedevices grouped with the same account.

If it is decided in operation 1437 that the plurality of devices arecapable of executing the command (YES in operation 1437), in operation1441, the server 750 may determine a feedback target device on the basisof a score of corresponding electronic devices. According to anembodiment, the server 750 may determine an electronic device having ahighest score among the plurality of electronic devices capable ofexecuting the command as a device for feedback of the user. For example,the server 750 may provide the feedback to the user to determine (orselect) a target device for executing a voice command by using Text ToSpeech (TTS) or other methods or the like. According to variousembodiments, when providing the feedback command, the server 750 maytransmit information (e.g., a device name (or a nickname)) for a devicethat can be selected based on the feedback command.

In operation 1443, the server 750 may transmit a feedback response(e.g., a third response) to a feedback target device. According to anembodiment, the server 750 may transmit a different response to wait fora response to remaining devices other than the feedback target device.

In operation 1445, the server 750 may receive a device designationcommand corresponding to a feedback from the feedback target device.

In operation 1447, the server 750 may determine a designated deviceamong devices grouped with the same account on the basis of the receiveddevice designation command, as a target device for executing a voicecommand.

FIG. 15 is a flowchart illustrating an operation of processing a voicecommand in an electronic device according to various embodiments of thedisclosure.

Referring to FIG. 15 , in operation 1501, the processor 480 of theelectronic device 400 may transmit to a server a request message foridentifying whether to execute the voice command. According to anembodiment, upon calculating the score, the processor 480 may transmitservice information including at least part of the calculated scoreinformation, device information, or a follow-up command to the server byusing a configured communication interface (e.g., the wirelesscommunication unit 410). According to various embodiments, the serviceinformation may include information for requesting the server toidentify whether to execute the voice command.

In operation 1503, the processor 480 may receive from the server aresponse corresponding to a request for identifying whether to executean operation.

In operation 1505, the processor 480 may decide whether a responsereceived from the server corresponds to a feedback response. Accordingto various embodiments, the processor 480 may decide, in sequence or inparallel, whether the response from the server is a feedback response, afirst response for executing a voice command, or a second response fortransitioning to a standby state.

If it is decided in operation 1505 that the received response is not thefeedback response (NO in operation 1505), proceeding to operation 1515,the processor 480 may perform operation 1515 and subsequent operations.

If it is decided in operation 1505 that the received response is thefeedback response (YES in operation 1505), in operation 1507, theprocessor 480 may output a feedback to request the user to designate atarget device for executing the voice command on the basis of thefeedback response. According to various embodiments, the processor 480may guide information on selectable devices on the basis of deviceinformation included in the feedback response. According to variousembodiments, the feedback output may be provided in a visual and/orauditory manner. According to an embodiment, a display may be used for arelated text output, or a speaker may be used for a related audiooutput. Alternatively, the display and the speaker may be used to outputboth of them together.

In operation 1509, the processor 480 may receive a command based onuser's utterance. According to an embodiment, a user may utter a voicefor designating (selecting) a specific device in response to a deviceselection feedback.

In operation 1511, upon receiving a command, the processor 480 maytransmit the input command to the server.

In operation 1513, the processor 480 may receive from the server aresponse corresponding to transmission of a command for designating adevice.

In operation 1515, the processor 480 may decide whether the receivedresponse corresponds to a first response for instructing to process thevoice command or a second response for instructing to transition to thestandby state.

If it is decided in operation 1515 that the received response is thefirst response (YES in operation 1515), in operation 1517, the processor480 may process the voice command. For example, the processor 480 mayexecute a voice recognition operation for user's utterance, and mayprocess an operation related to execution of the related function.

If it is decided in operation 1515 that the received response is thesecond response (NO in operation 1515), in operation 1519, the processor480 may transition to the standby state to wait for the voice callcommand. According to an embodiment, the electronic device 400 maytransition to a sleep state.

An example of processing a voice command on the basis of interworking ofan electronic device and a server has been described above according tovarious embodiments. Hereinafter, an example of processing a voicecommand on the basis of an electronic device without intervention of theserver will be described according to various embodiments.

FIG. 16 illustrates an operation of processing a voice in a systemaccording to various embodiments of the disclosure.

As shown in FIG. 16 , a case shown in FIG. 16 may be as case where afirst electronic device 1616, a second electronic device 1620, and athird electronic device 1630 are grouped based on a specific account,and the same voice call command is registered. In addition, FIG. 16 mayshow an example of processing a voice command based on interworkingbetween electronic devices.

Referring to FIG. 16 , in operation 1601, a user may utter a voice callcommand so that a voice command is executed in a specific electronicdevice. According to various embodiments, the user may utter the voicecall command and a follow-up command, and may separately utter thefollow-up command after waking up the electronic device on the basis ofthe voice call command.

In operation 1603, the first electronic device 1610, the secondelectronic device 1620, and the third electronic device 1630 may wake upon the basis of the voice call command. According to variousembodiments, the first electronic device 1610, the second electronicdevice 1620, and the third electronic device 1630 may simultaneouslywake up within a specific time range in sequence or in parallel at atime of recognizing the voice call command. According to variousembodiments, the first electronic device 1610, the second electronicdevice 1620, and the third electronic device 1630 may receive andrecognize the voice call command of the user through a microphone.

In operation 1605, the first electronic device 1610, the secondelectronic device 1620, and the third electronic device 1630 maycalculate a recognition score of the voice call command. In variousembodiments, the first electronic device 1610, the second electronicdevice 1620, and the third electronic device 1630 may calculate arecognition score for the voice call command of the user on the basis ofthe wake-up caused by the voice call command.

In operation 1607, the first electronic device 1610, the secondelectronic device 1620, and the third electronic device 1630 mayexchange the calculated score with other electronic devices. Accordingto an embodiment, the first electronic device 1610 may transmit a firstscore to the second electronic device 1620 and the third electronicdevice 1630. The first electronic device 1610 may receive a second scoreof the second electronic device 1620 and a third score of the thirdelectronic device 1630 respectively from the second electronic device1620 and the third electronic device 1630. For example, the electronicdevice 400 may transmit its score to other electronic devices, and mayacquire scores thereof from the other electronic device with the sameaccount.

In operation 1609, the first electronic device 1610, the secondelectronic device 1620, and the third electronic device 1630 maydetermine whether the device is a target device for executing a voicecommand on the basis of the score. According to an embodiment, the firstelectronic device 1610, the second electronic device 1620, and the thirdelectronic device 1630 may compare a score thereof and a score acquiredfrom a different electronic device to decide whether the device has ahighest score. According to an embodiment, FIG. 16 may show a case wherethe first electronic device 1610 has a highest score, and the firstelectronic device 1610 is determined as the target device.

In operations 1611, 1621, and 1631, the first electronic device 1610,the second electronic device 1620, and the third electronic device 1630may determine a target device or a transition to a standby state on thebasis of a decision result. According to an embodiment, in operation1611, the first electronic device 1610 may decide that it has a highestscore and thus may determine the first electronic device 1610 as atarget device for executing a voice command. According to an embodiment,in operations 1621 and 1631, the second electronic device 1620 and thethird electronic device 1630 may decide that their scores are lower thanthe score of the first electronic device 1610 and thus may determine towait for a next voice call command.

In operations 1613, 1623, and 1633, the first electronic device 1610,the second electronic device 1620, and the third electronic device 1630may execute a corresponding operation on the basis of the determinationresult. According to an embodiment, in operation 1613, the firstelectronic device 1610 may operate to execute a voice command. Forexample, the first electronic device 1610 may operate to perform arelated service based on the voice recognition (e.g., processing of thevoice command and executing of a function related to the voice command).According to an embodiment, in operations 1623 and 1625, the secondelectronic device 1020 and the third electronic device 1030 may operateto transition to the standby state.

FIG. 17 is a flowchart illustrating an operation of processing a voicecommand in an electronic device according to various embodiments of thedisclosure.

Referring to FIG. 17 , in operation 1701, the processor 480 of theelectronic device 400 may detect a voice call command. According to anembodiment, the electronic device 400 may recognize the voice callcommand through a microphone on the basis of a first processor (e.g., awake-up processing unit, a wake-up engine) for recognizing the voicecall command.

In operation 1703, the processor 480 may wake up in response todetection of the voice call command. According to an embodiment, theelectronic device 400 may wake up a second processor (e.g., a follow-upcommand recognition unit, a voice recognition engine, a service controlcircuit), on the basis of voice call command detection of the firstprocessor.

In operation 1705, the processor 480 may identify a different electronicdevice included in a group of the same account on the basis of thewake-up. According to an embodiment, the processor 480 may check whetherthere is an electronic device belonging to a group of which an accountis the same as its own score by referring to pre-set information (e.g.,a domain list).

In operation 1707, the processor 480 may decide whether there is thedifferent electronic device grouped with the same account on the basisof the identification result.

If it is decided in operation 1707 that the different electronic devicedoes not exist (NO in operation 1707), processing to operation 1721, theprocessor 480 may process operation 1721 and subsequent operations.

If it is decided in operation 1707 that the different electronic deviceexists (YES in operation 1707), in operation 1709, the processor 480 maycalculate a recognition score of the voice call command. According to anembodiment, the processor 480 may calculate a score by consideringvarious situations, as explained with reference to Equation 1 and FIG. 9.

In operation 1711, the processor 480 may exchange the calculated scorewith different electronic devices of the same account. For example, theelectronic device 400 may use a configured communication interface(e.g., the wireless communication unit 410) to transmit its score todifferent electronic devices of the same account and receive theirscores from the different electronic devices of the same account.According to an embodiment, upon calculating the score, the processor480 may use the configured communication interface (e.g., the wirelesscommunication unit 410) to perform an operation of exchanging thecalculated score information and device information with the differentelectronic device.

In operation 1713, the processor 480 may compare its score with a scorereceived from the different electronic device.

In operation 1715, the processor 480 may decide whether its scorecorresponds to a highest score on the basis of a comparison result.

If it is decided in operation 1715 that its score is not the highestscore (NO in operation 1715), proceeding to operation 1727, theprocessor 480 may perform operation 1727 and subsequent operations.

If it is decided in operation 1715 that its score is the highest score(YES in operation 1715), in operation 1717, the processor 480 may decidewhether there is a different electronic device of the same score.According to an embodiment, the processor 480 may decide whether thereis a score which is the same as its own score among scores of differentelectronic devices grouped with the same account.

If it is decided in operation 1717 that there is no different electronicdevice of the same score (NO in operation 1717), proceeding to operation1723, the processor 480 may perform operation 1723 and subsequentoperations.

If it is decided in operation 1717 that the different electronic deviceof the same score exists (YES in operation 1717), in operation 1719, theprocessor 480 may decide a priority. According to an embodiment, theremay be one or more different electronic devices having the same score.The processor 480 may decide whether to apply a priority according to apre-set priority. According to various embodiments, per-device priorityinformation may be set in a domain list. According to variousembodiments, the deciding of whether to apply the priority may bedetermined based on a different determination condition (e.g., afollow-up command (e.g., a device designation command, a domaincommand)) as shown in the drawings described below.

In operation 1721, on the basis of the decision result, the processor480 may decide whether the priority is higher compared to a differentelectronic device or whether the priority is lower compared to thedifferent electronic device.

If it is decided in operation 1721 that the priority is high (YES inoperation 1721), in operation 1723, the processor 480 may determine thatit is a target device for executing a voice command.

In operation 1725, the processor 480 may process the voice command inresponse to the determination of the target device. For example, theprocessor 480 may execute a voice recognition operation for user'sutterance, and may process an operation related to execution of therelated function.

If it is decided in operation 1721 that the priority is low (NO inoperation 1721), in operation 1727, the processor 480 may determine towait for the voice call command.

In operation 1729, the processor 480 may transition to the standby statein response to the determination of waiting. According to an embodiment,the processor 480 may allow the electronic device 400 to transition to asleep state to wait for the voice call command, and may not execute avoice recognition operation for the user's utterance.

FIG. 18 is a flowchart illustrating an example of determining a targetdevice for executing a voice command in an electronic device accordingto various embodiments of the disclosure.

Referring to FIG. 18 , in operation 1801, the processor 480 of theelectronic device 400 may detect a voice call command. According to anembodiment, the electronic device 400 may recognize the voice callcommand through a microphone on the basis of a first processor (e.g., awake-up processing unit, a wake-up engine) for recognizing the voicecall command.

In operation 1803, the processor 480 may wake up in response todetection of the voice call command. According to an embodiment, theelectronic device 400 may wake up a second processor (e.g., a follow-upcommand recognition unit, a voice recognition engine, a service controlcircuit), on the basis of voice call command detection of the firstprocessor.

In operation 1805, the processor 480 may analyze the voice call commandon the basis of the wake-up. According to an embodiment, the processor480 may decide a determination condition to be considered to determinewhether a device designation command (e.g., “phone”, etc.) is includedtogether with the voice call command, or whether a domain command (e.g.,“music playback”, etc.) is included, or the like. According to variousembodiments, the determination condition may include one or more of thedevice designation command and the domain command.

In operation 1807, the processor 480 may decide whether the devicedesignation command is included on the basis of an analysis result.

If it is decided in operation 1807 that the device designation commandis included (YES in operation 1807), in operation 1809, the processor480 may decide a designated device. For example, the processor 480 maydetermine whether the designated device is its electronic device 400according to the device designation command.

In operation 1811, the processor 480 may decide whether the device is atarget device for executing a voice command on the basis of the decisionresult. According to an embodiment, the processor 480 may decide whetherthe device designation command corresponds to device information (e.g.,a device name) indicating the electronic device 400.

If it is decided in operation 1811 that the device designation commandindicates the electronic device 400 (YES in operation 1811), inoperation 1813, the processor 480 may determine the electronic device400 as the target device for executing the voice command. According toan embodiment, if the device designated in the device designationcommand is its electronic device 400, the processor 480 may determinethe device as the target device for executing the voice command.

In operation 1815, the processor 480 may process the voice command inresponse to the determination of the target device. For example, theprocessor 480 may execute a voice recognition operation for user'sutterance, and may process an operation related to execution of therelated function.

If it is decided in operation 1811 that the device designation commandindicates a different electronic device (NO in operation 1811), inoperation 1817, the processor 480 may determine to wait for the voicecall command.

In operation 1819, the processor 480 may transition to the standby statein response to the determination of waiting. According to an embodiment,the processor 480 may allow the electronic device 400 to transition to asleep state to wait for the voice call command, and may not execute avoice recognition operation for the user's utterance.

If it is decided in operation 1807 that the device designation commandis not included (NO in operation 1807), in operation 1821, the processor480 may decide whether a domain command is included.

If it is decided in operation 1821 that the domain command is included(YES in operation 1821), in operation 1823, the processor 480 mayprocess a related operation on the basis of the domain command. Aspecific example for this will be described in detail with reference toFIG. 19 .

If it is decided in operation 1821 that the domain command is notincluded (NO in operation 1821), in operation 1825, the processor 480may process a related operation, such as determining of a target devicefor executing a voice command on the basis of a score of devices groupedwith the same account.

FIG. 19 is a flowchart illustrating an example of determining a targetdevice for executing a voice command in an electronic device accordingto various embodiments of the disclosure.

Referring to FIG. 19 , in operation 1901, if it is decided that a domaincommand is included, the processor 480 of the electronic device 400 mayanalyze a domain list.

In operation 1903, the processor 480 may identify one or more devicescapable of executing a command (function) based on the domain commandamong devices grouped with the same account, on the basis of an analysisresult of the domain list. According to various embodiments, if it isdecided that a command based on the domain command cannot be executed inthe electronic device 400, the processor 480 may provide control toautomatically transition to a standby state while omitting operationsdescribed below. In the example of FIG. 19 , the electronic device 400may be exemplified as the device capable of executing the command(function) based on the domain command.

In operation 1905, the processor 480 may decide whether a plurality ofdevices are capable of executing the command on the basis of theidentification result. For example, it may be decided whether there aredifferent electronic devices having the same domain among devicesgrouped with the same account.

If it is decided in operation 1905 that there is one device capable ofexecuting the command (NO in operation 1905), proceeding to operation1921, the processor 480 may perform operation 1921 and subsequentoperations. For example, the processor 480 may determine the electronicdevice 400 as the target device for executing the voice command.

If it is decided in operation 1905 that the plurality of devices arecapable of executing the command (YES in operation 1905), in operation1907, the processor 480 may determine a feedback target device on thebasis of a score of corresponding electronic devices. According to anembodiment, the processor 480 may determine an electronic device havinga highest score among the plurality of electronic devices capable ofexecuting the command as a device for feedback of the user. For example,the processor 480 may provide the feedback to the user to determine (orselect) a target device for executing a voice command by using Text ToSpeech (TTS) or other methods or the like.

In operation 1909, the processor 480 may decide whether the electronicdevice 400 is a feedback target device. According to an embodiment, ifthe electronic device 400 has a highest score, the processor 480 maydecide that the electronic device 400 is the feedback target device. Ifthe electronic device 400 does not have the highest score, the processor480 may decide that the electronic device 400 is not the feedback targetdevice.

If it is decided in operation 1909 that the electronic device 400 is thefeedback target device (YES in operation 1909), in operation 1911, theprocessor 480 may output a feedback. According to various embodiments,the processor 480 may guide information for a request to select any oneof selectable devices for executing a command. According to variousembodiments, the feedback output may be provided in a visual and/orauditory manner. According to an embodiment, a display may be used for arelated text output, or a speaker may be used for a related audiooutput. Alternatively, the display and the speaker may be used to outputboth of them together.

In operation 1913, the processor 480 may receive a response based onuser's utterance. According to an embodiment, a user may utter a voicefor designating (selecting) a specific device in response to a deviceselection feedback.

In operation 1915, upon receiving a response corresponding to thefeedback output, the processor 480 may decide whether the electronicdevice 400 is a target device for executing a voice command. Accordingto an embodiment, the processor 480 may decide whether the responsedesignates the electronic device 400 or designates a differentelectronic device.

If it is decided in operation 1915 that a user's response indicates adifferent electronic device (NO in operation 1915), in operation 1917,the processor 480 may transfer a response for instructing a designateddifferent electronic device to execute the voice command. According toan embodiment, the processor 480 may transmit a first response forinstructing a corresponding target device to execute the voice command.

In operation 1919, the processor 480 may transition to the standbystate. According to an embodiment, the processor 480 may allow theelectronic device 400 to transition to a sleep state to wait for thevoice call command, and may not execute a voice recognition operationfor the user's utterance.

If it is decided in operation 1915 that the user's response indicatesthe electronic device 400 (YES in operation 1915), in operation 1921,the processor 480 may determine the electronic device 400 as the targetdevice for executing the voice command.

In operation 1923, the processor 480 may transfer a response forinstructing a different electronic device to transition to the standbystate in response to the determination of the target device. Accordingto an embodiment, the processor 480 may transmit a second response forinstructing the different electronic device to transition to the standbystate for waiting for the voice call command.

In operation 1925, the processor 480 may process the voice command inresponse to the determination of the target device. For example, theprocessor 480 may execute a voice recognition operation for user'sutterance, and may process an operation related to execution of therelated function.

If it is decided in operation 1909 that the electronic device 400 is notthe feedback target device (NO in operation 1909), in operation 1927,the processor 480 may wait to receive a response from a differentelectronic device. According to an embodiment, the processor 480 mayexist in the standby state (e.g., a time-based temporary standby staterather than waiting in a sleep state) for receiving a response during apre-set specific time. According to another embodiment, the processor480 may process an automatic transition to the standby state (e.g., asleep state) at the expiry of the specific time set in the temporarystandby state.

In operation 1929, the processor 480 may receive a response from adifferent electronic device.

In operation 1931, the processor 480 may decide whether the receivedresponse corresponds to a first response for instructing to process thevoice command or a second response for instructing to transition to thestandby state.

If it is decided in operation 1931 that the received response is thefirst response (YES in operation 1931), in operation 1933, the processor480 may determine the electronic device 400 as the target device forexecuting the voice command.

In operation 1935, the processor 480 may process the voice command inresponse to the determination of the target device. For example, theprocessor 480 may execute a voice recognition operation for user'sutterance, and may process an operation related to execution of therelated function.

If it is decided in operation 1931 that the received response is thesecond response (NO in operation 1931), in operation 1937, the processor480 may determine to wait for the voice call command.

In operation 1939, the processor 480 may transition to the standby statein response to the determination of waiting. According to an embodiment,the processor 480 may allow the electronic device 400 to transition to asleep state to wait for the voice call command, and may not execute avoice recognition operation for the user's utterance.

As described above, a method of operating an electronic device accordingto various embodiments may include waking up on the basis of detectionof a voice call command, calculating a score related to recognition ofthe voice call command, sharing the score with an external device,deciding whether to execute a voice command on the basis of the score,and processing the voice command on the basis of the decision result.

According to various embodiments, the calculating of the score mayinclude calculating the score, based at least in part on a distance to auser, an environment of the electronic device, a location of theelectronic device, a domain of the electronic device, or a voicerecognition rate.

According to various embodiments, the deciding of whether to execute thevoice command may include transmitting the calculated score to theexternal device, receive a response determined based on the score fromthe external device, and determining whether to process the voicecommand on the basis of the received response.

According to various embodiments, the processing of the voice commandmay include processing the voice command if the response from theexternal device is a first response for instructing to process the voicecommand, and transitioning to a standby state if the response from theexternal device is a second response for instructing to transition tothe standby state.

According to various embodiments, the processing of the voice commandmay include outputting a related feedback if the response from theexternal device is a feedback response for requesting a user to select adevice, and acquiring a device designation command of the user for thefeedback.

According to various embodiments, the deciding of whether to execute thevoice command may include deciding a score of the electronic device anda score acquired by the external device, and if the score of theelectronic device is high, determining the device as a target device forprocessing the voice command.

According to various embodiments, the deciding of whether to execute thevoice command may include determining whether to process the voicecommand on the basis of a device designation command or a domaincommand.

According to various embodiments, the deciding of whether to execute thevoice command may include determining a feedback target device on thebasis of the score if a plurality of electronic devices include arelated domain among devices grouped with the same account on the basisof the domain command, acquiring device selection performed by a user onthe basis of the determined feedback target device, and determining anelectronic device corresponding to the device selection performed by theuser as a target device for executing a voice command.

According to various embodiments, the method may further includeacquiring the determination of the feedback target device and thedetermination of the target device from the external device.

According to various embodiments, the external device may determine atarget device for executing the voice command, based at least in part ona score of electronic devices grouped with the same account, a devicedesignation command, or a domain command, transmit a first response forinstructing to process the voice command to the determined targetdevice, and transmit a second response for instructing a differentelectronic device other than the target device among the electronicdevices to transition to a standby state. The electronic device mayprocess the voice command on the basis of receiving of the firstresponse from the external device, or transition to the standby state onthe basis of receiving of the second response.

According to various embodiments, the voice command may be determined tobe processed by any one of a plurality of electronic devices which wakeup on the basis of the voice call command and which are grouped with thesame account.

Various embodiments of the disclosure disclosed in the specification andthe drawing are merely a specific example presented for clarity and arenot intended to limit the scope of the disclosure. Therefore, inaddition to the embodiments disclosed herein, various changes in formsand details made without departing from the technical concept of thedisclosure will be construed as being included in the scope of thedisclosure.

What is claimed is:
 1. An electronic device comprising: a microphone;communication circuitry; a memory; and a processor operatively coupledto the microphone and the memory, wherein the processor is configuredto: detect a voice command including a voice call command using themicrophone; obtain a score related to the voice call command; controlthe communication circuitry to transmit a request including informationon the score to a server; when the electronic device is determined as afeedback target device based on the score, receive a first response tothe request from the server using the communication circuitry; inresponse to the first response being a feedback command, output afeedback for requesting a user to select a target device performing thevoice command, wherein the feedback includes guide information onselectable devices as the target device; in response to the feedback,detect a speech of the user selecting the target device using themicrophone; and process an operation related to the voice command basedon the detected speech, the processing of the operation comprising:based on the detected speech of the user indicating the electronicdevice, determining the electronic device as the target device,transferring a signal for instructing a different electronic device totransition to a standby state, and processing the voice command; andbased on the detected speech of the user indicating the differentelectronic device, transferring a signal for instructing the differentelectronic device to execute the voice command, and transition to thestandby state to wait for the voice call command.
 2. The electronicdevice of claim 1, wherein the processor comprises: a first processorconfigured to recognize the voice call command; and a second processorconfigured to wake up based on detection of the voice call command ofthe first processor and to process voice recognition and the operationrelated to the voice command.
 3. The electronic device of claim 1,wherein the processor is configured to obtain the score, based at leastin part on a distance to a user, an environment of the electronicdevice, a location of the electronic device, a domain of the electronicdevice, or a voice recognition rate.
 4. The electronic device of claim1, wherein the processor is configured to: control the communicationcircuitry to transmit the speech to the server; receive a secondresponse responsive to the speech from the server using thecommunication circuitry; and determine whether to process the voicecommand based on the received second response.
 5. The electronic deviceof claim 4, wherein the processor is configured to: process the voicecommand based on the second response from the server being a responsefor instructing to process the voice command; and transition to astandby state based on the second response from the server being aresponse for instructing to transition to the standby state.
 6. A methodof providing a voice service in an electronic device, the methodcomprising: detecting a voice command including a voice call commandusing a microphone of the electronic device; obtaining a score relatedto the voice call command; transmitting, using communication circuitryof the electronic device, a request including information on the scoreto a server; receiving a first response to the request from the serverusing the communication circuitry, when the electronic device isdetermined as a feedback target device based on the score; in responseto the first response being a feedback command, outputting a feedbackfor requesting a user to select a target device performing the voicecommand, wherein the feedback includes guide information on selectabledevices as the target device; in response to the feedback, detecting aspeech of the user selecting the electronic device as the target deviceusing the microphone; and processing on operation related to the voicecommand based on the detected speech, the processing operationcomprising: based on the detected speech of the user indicating theelectronic device, determining the electronic device as the targetdevice, transferring a signal for instructing a different electronicdevice to transition to a standby state, and processing the voicecommand; and based on the detected speech of the user indicating thedifferent electronic device, transferring a signal for instructing thedifferent electronic device to execute the voice command, and transitionto the standby state to wait for the voice call command.
 7. The methodof claim 6, wherein the deciding of whether to execute the voice commandcomprises: transmitting the speech to the server; receiving a secondresponse responsive to the speech from the server using thecommunication circuitry; and determining whether to process the voicecommand based on the received second response.
 8. The method of claim 7,wherein the processing of the voice command comprises: processing thevoice command based on the second response from the server being aresponse for instructing to process the voice command; and transitioningto a standby state based on the second response from the server being aresponse for instructing to transition to the standby state.
 9. Themethod of claim 6, wherein the obtaining of the score comprises:obtaining the score based at least in part on a distance to a user, anenvironment of the electronic device, a location of the electronicdevice, a domain of the electronic device, or a voice recognition rate.